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Why GAO Did This Study 

Air carriers remain a front-line 
defense against acts of terrorism 
that target the nation's civil 
aviation system. A key 
responsibility of air carriers is to 
check passengers' names against 
terrorist watch-list records to 
identify persons who should be 
prevented from boarding (the No 
Fly List) or who should undergo 
additional security scrutiny (the 
Selectee List). Eventually, the 
Transportation Security 
Administration (TSA) is to assume 
this responsibility through its 
Secure Flight program. However, 
due to program delays, air carriers 
retain this role. You asked GAO to 
review domestic air carriers' 
watch-list-matching processes. 
GAO examined (1) the watch-list- 
matching requirements air carriers 
must follow that have been 
established by TSA, and (2) the 
extent to which TSA has assessed 
air carriers' compliance with these 
requirements. GAO reviewed TSA's 
security directives, internal 
guidance used by TSA's inspectors 
to assess air carriers' compliance 
with requirements, and inspection 
results, as well as interviewed staff 
from 14 of 95 domestic air carriers 
(selected to reflect a range in 
operational sizes). This report is 
the public version of a restricted 
report (GAO-08-453SU) issued in 
July 2008. 



What GAO Recommends 



GAO is not making any 
recommendations because TSA 
initiated actions in April 2008 to 
strengthen watch-list-matching 
requirements and its oversight of 
air carriers' implementation of 
these requirements. 

To view the full product, including the scope 
and methodology, click on GAO-08-992. 
For more information, contact Cathleen A. 
Berrick at (202) 512-3404 or 
berrickc@gao.gov. 



What GAO Found 

TSA's requirements for domestic air carriers to conduct watch-list matching 
include a requirement to identify passengers whose names are either identical 
or similar to those on the No Fly and Selectee lists. Similar-name matching is 
important because individuals on the watch list may try to avoid detection by 
making travel reservations using name variations. According to TSA's Office 
of Intelligence, there have been incidents of air carriers failing to identify 
potential matches by not successfully conducting similar-name matching. 
However, until revisions were initiated in April 2008, TSA's security directives 
did not specify what types of similar-name variations were to be considered 
by air carriers. Thus, in interviews with 14 air carriers GAO found inconsistent 
approaches to conducting similar-name matching. Due to such inconsistency, 
a passenger could be identified as a match by one air carrier and not by 
another. In addition, not every air carrier reported conducting similar name 
comparisons. Further, in January 2008, TSA conducted an evaluation of air 
carriers and found deficiencies in their capability to conduct similar-name 
matching. Shortly thereafter, in April 2008, TSA revised the No Fly List 
security directive to specify a baseline capability for conducting watch-list 
matching, and TSA reported that it planned to similarly revise the Selectee 
List security directive. Because the baseline capability requires that air 
carriers compare only the types of name variations specified in the directive, 
TSA recognizes that the new baseline capability will not address all 
vulnerabilities. However, TSA emphasized that establishing the baseline 
capability should improve air carriers' performance of watch-list matching 
and, in TSA's view, is the best interim solution pending the implementation of 
Secure Flight. 

TSA has undertaken various efforts to assess domestic air carriers' 
compliance with watch-list matching requirements; however, until 2008, TSA 
had conducted limited testing of air carriers' similar-name-matching 
capability. In 2005, for instance, TSA conducted an evaluation to determine 
whether air carriers had the capability to identify names that were identical — 
but not similar — to those on the No Fly List. Also, regarding regularly 
conducted inspections, TSA's guidance did not specifically direct inspectors 
to test air carriers' similar-name-matching capability, nor did the guidance 
specify the number or types of name variations to be assessed. Records in 
TSA's database for regular inspections conducted during 2007 made reference 
to name-match testing in 61 of the 1,145 watch-list-related inspections that 
GAO reviewed. Without criteria or standards for air carriers to follow in 
comparing name variations, TSA did not have a uniform basis for assessing 
compliance and addressing deficiencies. However, during the course of GAO's 
review and prompted by findings of the evaluation conducted in January 2008, 
TSA reported that its guidance for inspectors would be revised to help ensure 
air carriers' compliance with security directives. Although TSA has plans to 
strengthen its oversight of air carriers' compliance with the revised security 
directives, it is too early to assess the extent of such oversight since TSA's 
efforts are ongoing and not completed. 
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September 9, 2008 
Congressional Committees 

Currently, more than 6 years after the terrorist attacks on September 11, 
2001, air carriers remain a front-line defense against acts of terrorism that 
target the nation's civil aviation system. A key aspect of air carriers' 
security responsibilities is to conduct preboarding checks of all 
passengers' personal information against terrorist watch-list records that 
contain information on thousands of individuals with known or potential 
links to terrorism. This process, referred to hereafter as watch-list 
matching, involves comparing passenger data — most prominently name 
and date of birth — against the No Fly List to identify individuals who 
should be prevented from boarding an aircraft, and against the Selectee 
List to identify individuals who must undergo enhanced screening at the 
checkpoint prior to boarding. 1 

The Transportation Security Administration (TSA) requires that domestic 
air carriers operating to, from, and within the United States conduct 
watch-list matching. 2 Data compiled by TSA's Office of Intelligence 
indicate that, at times, these air carriers have failed to identify individuals 
who are on the No Fly List. For instance, for the 3-year period from 
January 2005 through December 2007, TSA documented several known 
incidents involving individuals on the No Fly List who, because of failures 
of domestic air carriers' watch-list-matching processes, were allowed to 



1 Watch-list matching is one of two TSA-mandated prescreening processes conducted by air 
carriers. The other mandated prescreening activity is the Computer Assisted Passenger 
Prescreening System, discussed later this report, which does not involve matching 
passenger information against the No Fly and Selectee lists. These lists contain applicable 
records from the Terrorist Screening Center's consolidated database of known or 
appropriately suspected terrorists. See GAO, Terrorist Watch List Screening: 
Recommendations to Promote a Comprehensive and Coordinated Approach to Terrorist- 
Related Screening, GAO-08-253T (Washington, D.C.: Nov. 8, 2007). 

2 The number of domestic air carriers has varied over time, for example, from 95 in 2005 to 
about 70 in 2007. For the purposes of this report, domestic air carriers are those with 
operations based in the United States that maintain full security programs in accordance 
with 49 C.F.R. part 1544. Foreign air carriers — air carriers with operations based outside 
the United States — must also comply with U.S. security regulations, including applicable 
requirements for watch-list matching, when operating flights to or from the United States in 
accordance with 49 C.F.R. part 1546. Both domestic and foreign air carriers may conduct 
international flights to and from the United States; however, these operations are outside 
the scope of this report. 
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board international flights traveling to or from the United States. 3 Data for 
these types of incidents, referred to as false negative watch-list-matching 
results, generally are not available for domestic flights — that is, domestic 
air carrier operations between two points within the United States or its 
territories. 4 Nevertheless, because the requirements for air carriers to 
conduct watch-list matching are generally the same irrespective of the 
departure or arrival location, false negative incidents may be occurring on 
domestic flights if watch-listed individuals attempt to fly domestically. 

At present, domestic air carriers generally conduct watch-list matching in 
accordance with requirements that TSA sets forth in security directives — a 
regulatory tool through which TSA may impose security measures on a 
regulated entity, in this case air carriers, generally in response to an 
immediate or imminent threat. 5 For example, security directives require 
that air carriers execute comparisons of passenger information with No 
Fly and Selectee list information within 24 hours of a flight's scheduled 
departure. TSA also has responsibility for overseeing how air carriers 
implement the requirements set forth in security directives. Critical to this 
effort are the agency's aviation security inspectors, who oversee air carrier 
efforts at air carriers' corporate security offices (principal security 
inspectors) and at airport locations (transportation security inspectors). 

As required by law, TSA is to take over from air carriers the function of 
matching passenger information to the No Fly and Selectee lists for 
domestic flights. 6 Since 2003, we have been assessing TSA's efforts to 



3 See GAO, Terrorist Watch List Screening: Opportunities Exist to Enhance Management 
Oversight, Reduce Vulnerabilities in Agency Screening Processes, and Expand Use of the 
List, GAO-08-110 (Washington, D.C.: Oct. 11, 2007). We reported that TSA's Office of 
Intelligence documented various incidents (for the period January 1, 2005, through June 3, 
2007) in which air carriers — both domestic and foreign — allowed individuals on the No Fly 
List to board international flights traveling to or from the United States. Several of these 
incidents involved flights of domestic air carriers. We asked TSA's Office of Intelligence to 
identify any additional incidents in which a No Fly listed individual flew on a domestic air 
carrier for the period June 4, 2007, through December 31, 2007, and TSA identified no 
additional incidents occurring within this time period. 

4 This issue of false negatives is addressed later in this report. 

° See, e.g., 49 C.F.R. § 1544.305. Although generally issued in response to an immediate or 
imminent threat, security directives may be effective for an indefinite duration if TSA 
determines that a continuing need for such measures exists. In some cases, aviation-related 
measures implemented through a security directive have been discontinued, amended, or 
incorporated into air carrier security programs. 

6 See 49 U.S.C. 44903(j)(2)(C). 
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develop such a watch-list-matching program, currently known as Secure 
Flight, and have reported that significant challenges, including the need to 
follow a more structured systems development approach and to fully 
address how the program would protect passengers' privacy rights, have 
delayed its implementation. 7 In April 2008, we reported that TSA has made 
significant progress in developing Secure Flight, but that challenges 
remained in a number of areas, including the need to develop more robust 
cost and schedule estimates. 8 We are continuing to review TSA's 
development and implementation of Secure Flight in response to requests 
from the U.S. Senate (Committee on Commerce, Science, and 
Transportation, and its Subcommittee on Aviation Operations, Safety, and 
Security; Committee on Appropriations, Subcommittee on Homeland 
Security; Committee on Homeland Security and Governmental Affairs; and 
Committee on the Judiciary) and the U.S. House of Representatives 
(Committee on Transportation and Infrastructure, Committee on 
Homeland Security, and the Committee on Oversight and Government 
Reform). In addition, the Consolidated Appropriations Act, 2008, requires 
that we report to the Committees on Appropriations of the Senate and 
House of Representatives on the Department of Homeland Security's 
(DHS) certification of 10 conditions outlined in section 522(a) of the 
Department of Homeland Security Appropriations Act, 2005, related to the 
development and implementation of the Secure Flight program. 9 The 
report is to be submitted 90 days after the DHS's Secretary certifies that all 
10 conditions have been successfully met. 

Pending Secure Flight's implementation, air carriers will continue to have 
primary responsibility for the watch-list-matching function. In conjunction 
with our ongoing evaluation of Secure Flight, we testified in June 2006 that 
due to delays and uncertainty surrounding Secure Flight's implementation, 
some air carriers were enhancing their watch-list-matching processes. We 



' GAO, Aviation Security: Computer-Assisted Passenger Prescreening System Faces 
Significant Implementation Challenges, GAO-04-385 (Washington, D.C.: Feb. 13, 2004); 
Aviation Security: Management Challenges Remain for the Transportation Security 
Administration's Secure Flight Program, GAO-06-864T (Washington, D.C.: June 14, 2006); 
and Aviation Security: Transportation Security Administration Has Strengthened 
Planning to Guide Investments in Key Aviation Security Programs, but More Work 
Remains, GAO-08-456T (Washington, D.C.: Feb. 28, 2008). 

8 GAO, Transportation Security: Efforts to Strengthen Aviation and Surface 
Transportation Security Continue to Progress, but More Work Remains, GAO-08-651T 
(Washington, D.C.: Apr. 15, 2008). 

9 See Pub. L. No. 110-161, Div. E, § 513, 121 Stat. 1844, 2072-73 (2007). 
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further identified that these improvements, though beneficial to the 
respective air carrier's operations, could further exacerbate differences 
that currently exist among the various air carriers, and could result in 
varying levels of effectiveness across air carriers in matching passenger 
information to the No Fly and Selectee lists. 10 

Due to the importance of identifying passengers who may pose a threat to 
commercial aviation, we were asked to review the current processes that 
domestic air carriers use to conduct watch-list matching for domestic 
flights. 11 Accordingly, this report addresses the following questions: 

• What are TSA's requirements for domestic air carriers to conduct 
watch-list matching for domestic flights? 

• To what extent has TSA assessed domestic air carriers' compliance 
with watch-list-matching requirements? 

This report is a public version of the restricted report (GAO-08-453SU) that 
we provided to you on July 10, 2008. DHS and TSA deemed some of the 
information in the restricted report as Sensitive Security Information, 
which must be protected from public disclosure. Therefore, this report 
omits this information, such as the specific details associated with the 
current processes that domestic air carriers use to conduct watch-list 
matching. Although the information provided in this report is more limited 
in scope, it addresses the same principal questions as the restricted report. 
Also, the overall methodology used for both reports is generally the same. 

To determine TSA's requirements for matching passenger information 
against the No Fly and Selectee lists for domestic flights, we reviewed 
TSA's security directives, policies, and other guidance applicable to watch- 
list matching. We also interviewed officials at TSA's Office of 
Transportation Sector Network Management, Office of Security 



GAO, Aviation Security: Management Challenges Remain for the Transportation 
Security Administration's Secure Flight Program, GAO-06-864T (Washington, D.C.: June 
14, 2006). 

11 We are conducting this review in response to requests from the House of Representatives 
(Committee on Transportation and Infrastructure, Committee on Homeland Security, and 
Committee on Oversight and Government Reform). These requesters asked that we review 
the current passenger prescreening system in conjunction with our ongoing work related to 
TSA's progress with Secure Flight. In addition, we are reporting on this issue to the U.S. 
Senate requesters and the mandate committees associated with our Secure Flight work. 
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Operations, Office of Intelligence, and Office of Chief Counsel. We also 
reviewed key policy documents for Secure Flight, as well as our most 
recent reports and testimonies on the program to determine the planned 
matching process. In addition, to identify the composition and use of the 
No Fly and Selectee lists, we interviewed officials with the Department of 
Justice, Federal Bureau of Investigation's (FBI) Terrorist Screening 
Center, which has responsibility for managing the use of terrorist 
information in screening processes. 12 We also contacted officials from a 
federally sponsored working group on identity matching to discuss the 
challenges associated with name-based matching. Moreover, to 
understand how air carriers have responded to watch-list-matching 
requirements, we conducted telephone interviews with officials from 14 
domestic air carriers. 13 Our selection of air carriers was based, in part, on 
operational size with the goal of obtaining a range of sizes based on 
operating revenue. For example, the Department of Transportation 
classifies eight of the air carriers in our review as major air carriers that 
provide service to locations across the nation and, with the exception of 
one air carrier, around the world. 14 The remaining six air carriers had 
comparatively smaller business operations that generally provided service 
covering a geographical area, such as the Pacific Northwest, or commuter 
service. 15 Although the 14 air carriers we spoke with represent a range in 
the types of air carriers that conduct watch-list matching, and, according 
to our calculations, accounted for approximately 70 percent of all 
passengers that boarded domestic flights in 2005, the results of our 
telephone interviews are not generalizable to the domestic operations of 



Pursuant to Homeland Security Presidential Directive 6, dated September 16, 2003, the 
Terrorist Screening Center — an entity that has been operational since December 2003 
under the administration of the FBI — was established to develop and maintain the U.S. 
government's consolidated terrorist screening database (the watch list) and to provide for 
the use of watch-list records during security-related screening processes. 

13 All 14 air carriers we interviewed operate under full security programs in accordance 
with 49 C.F.R. part 1544 and conduct watch-list matching in accordance with the No Fly 
and Selectee list security directives issued by TSA. 

14 The Department of Transportation groups U.S.-based air carriers according to their 
operating revenue. In the 2005 groupings, each of the "major" air carriers had over 
$1 billion in operating revenue. 

15 Of these six, the Department of Transportation's 2005 revenue groupings identified three 
as "national" air carriers, with each having over $100 million to $1 billion in operating 
revenue, and one as a "regional" air carrier, with $100 million or less in operating revenue. 
The other two air carriers were not included in the department's revenue groupings, given 
the small scale of operations, but were identified by the department as air carriers that 
provide commuter service. Major air carriers have over $1 billion in operating revenue. 



Page 5 



GAO-08-992 Aviation Security and Watch List Matching 



all domestic air carriers. However, our selection allowed us to understand 
how watch-list matching was performed for the majority of passengers 
flying domestically in 2005. In addition, although our work summarizes the 
14 air carriers' watch-list-matching capabilities as described to us in 
interviews, we did not independently verify each air carrier's reported 
method of implementation to determine the reliability of the data. 

To determine the extent to which TSA has assessed domestic air carriers' 
compliance with watch-list-matching requirements in the No Fly and 
Selectee list security directives, 16 we first assessed TSA's inspection 
process, including the focus of inspections and inspection methods. We 
also examined TSA's national inspection plans and related guidance and 
policy documents. Further, at TSA headquarters, we interviewed officials 
responsible for developing and implementing inspection guidance and 
compiling and analyzing inspection results. Specifically, we interviewed 
representatives from the Office of Security Operations and the Office of 
Transportation Sector Network Management. We analyzed the results of 
both regular inspections (i.e., inspections conducted in conjunction with 
annual inspection plans) and nonroutine watch-list-related inspections 
that TSA conducted. For instance, we analyzed regular watch-list-related 
inspections that TSA conducted during fiscal year 2007 to ensure that air 
carriers were in compliance with applicable requirements. Although we 
concluded that these regular inspection data were sufficiently reliable for 
the purposes of this report, we have concerns about the potential for error 
based on TSA's process for querying its inspection database (we discuss 
these concerns in more detail in app. I). To assess data reliability, we 
performed electronic testing, discussed the data system and any data 
inconsistencies we found with knowledgeable TSA officials, and reviewed 
existing information about the data system. We also reviewed results from 
a special emphasis assessment that TSA conducted in 2005, and a special 
emphasis inspection it conducted in January 2008, both of which 
addressed air carriers' capability to conduct watch-list matching. 17 We 
determined that the sampling and related procedures used for the special 
emphasis assessment were insufficient for providing a reliable estimate of 



The No Fly and Selectee list security directives also address the screening of air carrier 
employees against the No Fly and Selectee lists, but our scope was confined to the 
passenger-specific prescreening requirements in the security directives. 

17 Special emphasis assessments and special emphasis inspections are nonroutine activities 
undertaken at the direction of TSA headquarters. According to TSA, a special emphasis 
assessment addresses a vulnerability that generally is not tied to a regulation, while a 
special emphasis inspection is tied to a regulatory requirement. 
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the success rate of all attempted matches by air carriers. We did not assess 
the initial data TSA provided in February 2008 for the special emphasis 
inspection it conducted the previous month. 18 

We conducted this performance audit from July 2006 to September 2008 in 
accordance with generally accepted government auditing standards. Those 
standards require that we plan and perform the audit to obtain sufficient, 
appropriate evidence to provide a reasonable basis for our findings and 
conclusions based on the audit objectives. We believe that the evidence 
obtained provides a reasonable basis for our findings and conclusions 
based on the audit objectives. More details about the scope and 
methodology of our work are presented in appendix I. 



RGSllltS in Brief TSA has issued two security directives (one for the No Fly List and 

, another for the Selectee List) that delineate requirements related to air 

carrier watch-list matching, including the identification of passengers with 
names similar to those on the lists. Identifying passengers with names 
similar to those on the No Fly and Selectee lists — a process TSA refers to 
as similar-name matching — is a critical component of watch-list matching 
because individuals may travel using abbreviated name forms or other 
variations of their names. Therefore, searching for only an exact match of 
the passenger's name may not result in identifying all watch-listed 
individuals. There have been incidents, according to TSA's Office of 
Intelligence, of air carriers failing to identify potential matches by not 
successfully conducting similar-name matching. Before revisions to the 
security directives were initiated in 2008, TSA expected air carriers to find 
similar names but provided no specificity on the extent to which air 
carriers should make these comparisons. The 14 air carriers we 
interviewed reported implementing varied approaches to similar-name 
matching. Because air carriers used different approaches, a passenger 
could be identified as a match to a watch-list record by one carrier and not 
by another carrier, which results in uneven effectiveness of watch-list 
matching. Generally, TSA had been aware that air carriers were not using 
equivalent processes to compare passenger names with names on the No 
Fly and Selectee lists. However, in early 2008 the significance of such 



In September 2008, TSA provided us the results of a special emphasis assessment 
(conducted during May 2008) of seven air carriers' compliance with new requirements in 
the No Fly List security directive, which was revised in April 2008 to specify a baseline 
capability for conducting watch-list matching. This special emphasis assessment is 
discussed later in this report. 
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differences was crystallized during the course of our review and following 
TSA's special emphasis inspection of air carriers' name-matching 
capability. On the basis of these inspection results, TSA issued a revised 
security directive governing the use of the No Fly List in April 2008 to 
establish a baseline capability for similar-name matching to which all air 
carriers must conform. Also, TSA announced that it plans to revise the 
Selectee List security directive to similarly require the new baseline 
capability. 19 According to TSA officials, the new baseline capability is 
intended to improve the effectiveness of watch-list matching, particularly 
for those air carriers that did not compare the types of name variations 
specified by the new baseline capability or that compared none at all. 
However, TSA officials noted that the new baseline is not intended to 
address all possible types of name variations and the related security 
vulnerabilities. Agency officials explained that based on their analysis of 
the No Fly and Selectee lists and interviews with intelligence community 
officials, the newly established baseline covers the most critical types of 
name variations. TSA officials further stated that this is an interim solution 
that will strengthen security while not requiring air carriers to invest in 
significant modifications to their watch-list-matching processes, given 
TSA's expected implementation of Secure Flight beginning in 2009. These 
officials added that when implemented, Secure Flight will be better able to 
use passenger names and other identifying information to more accurately 
match passengers to the subjects of watch-list records. 

TSA has undertaken various efforts to assess domestic air carriers' 
compliance with watch-list-matching requirements in the No Fly and 
Selectee list security directives; however, until 2008, TSA had conducted 
limited testing of air carriers' similar-name-matching capability. In 2005, 
for instance, TSA conducted a special emphasis assessment that focused 
on air carriers' capability to prescreen passengers for exact-name matches 
with the No Fly List, but did not address the air carriers' capability to 
conduct similar-name comparisons. Regarding inspections conducted as 
part of regular inspection cycles, TSA's guidance establishes that 
regulatory requirements encompassing critical layers of security need 
intensive oversight, and that testing is the preferred method for validating 
compliance. However, before being revised in 2008, TSA's inspection 



In September 2008, TSA informed us that the revised Selectee List security directive was 
still in the agency's internal clearance process but did not provide us a targeted issuance 
date. 
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guidelines (called PARIS prompts) 20 for watch-list-related inspections were 
broadly stated and did not specifically direct inspectors to test air carriers' 
similar-name-matching capability. Moreover, TSA's guidance provided no 
baseline criteria or standards regarding the number or types of such 
variations that must be assessed. In response to our inquiry, 6 of TSA's 9 
principal security inspectors told us that their assessments during annual 
inspection cycles have not included examining air carriers' capability to 
conduct certain basic types of similar-name comparisons. Also, in 
reviewing documentation of the results of the most recent inspection cycle 
(fiscal year 2007), we found that available records in TSA's database made 
references to name-matching tests in 6 of the 36 watch-list-related 
inspections that principal security inspectors conducted, and in 55 of the 
1,109 inspections that transportation security inspectors conducted. 21 
Without baseline criteria or standards for air carriers to follow in 
conducting similar-name comparisons, TSA has not had a uniform basis 
for assessing compliance. Further, without routinely and uniformly testing 
how effectively air carriers are conducting similar-name matching, TSA 
may not have had an accurate understanding of the quality of air carriers' 
watch-list-matching processes. However, TSA began taking corrective 
actions during the course of our review and after it found deficiencies in 
the capability of air carriers to conduct similar-name matching during a 
January 2008 special emphasis inspection. 22 More specifically, following 
the January 2008 inspection, TSA officials reported that TSA immediately 
began working with individual air carriers to address deficiencies. Also, 
officials reported that, following the issuance of TSA's revised No Fly List 
security directive in April 2008, the agency had plans to assess air carriers' 
progress in meeting the baseline capability specified in the new security 
directive after 30 days, and that the annual inspection plan for 
transportation security inspectors would be revised to help ensure 
compliance by air carriers with requirements in the new security directive. 
In September 2008, TSA provided us with results from a May 2008 special 
emphasis assessment of seven air carriers' compliance with the revised No 



PARIS is the acronym for the Performance and Results Information System, which is 
TSA's inspections database. This database assists TSA management by providing factual 
and analytical information on the compliance of TSA-regulated entities. There are 
approximately 1,700 PARIS prompts, which serve as guidelines for TSA inspectors. 

21 According to TSA data, these 1,145 watch-list-related inspections (36 plus 1,109) covered 
60 domestic air carriers, and most of the air carriers were inspected multiple times. 

22 TSA reported that the January 2008 special emphasis inspection covered 52 domestic air 
carriers and 31 foreign air carriers. 
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Fly List security directive. Although the details of this special emphasis 
assessment are classified, TSA generally characterized the results as 
positive. Further, the TSA officials noted that the agency's internal 
handbook — which provides guidance to transportation security inspectors 
on how to inspect air carriers' performance of various requirements, 
including watch-list-matching requirements — was being revised and was 
expected to be released later this year. Thus, the TSA officials stated that 
the new inspection guidance would be used in conjunction with the 
nationwide regulatory activities plan for fiscal year 2009. While these 
actions and plans are positive developments, it is too early to determine 
the extent to which TSA will assess air carriers' compliance with watch- 
list-matching requirements moving forward since these efforts are still 
underway. 

We provided a draft of our restricted report to DHS and the Department of 
Justice for review and comment. DHS had no comments. The Department 
of Justice provided technical comments to the restricted version of this 
report, which we incorporated where appropriate. 



Background. uses a ^ a ^ ere( ^ s y s t em °f defense to secure civil aviation whereby 

" additional layers provide security when any one security measure may fail. 

Watch-list matching is one such layer of defense. Air carriers began 
checking passenger names against government-supplied terrorist watch 
lists (compiled by the FBI and distributed by the Federal Aviation 
Administration) in the early 1990s. After the attacks of September 11, 2001, 
and the subsequent establishment of TSA during the same year, primary 
responsibility for civil aviation security, including overseeing the watch- 
list-matching process, fell to TSA. 23 The Aviation and Transportation 
Security Act, enacted in November 2001, requires that a system be used to 
evaluate all passengers before they board an aircraft and ensure that 
selected individuals and their carry-on and checked baggage are 
adequately screened. 24 TSA fulfilled this mandate by continuing to require 



In accordance with 49 U.S.C. § 114(h), TSA adopted policies and procedures for ensuring 
that air carriers use information from government agencies to identify individuals on 
passenger lists who may be a threat to civil aviation or national security and, if such an 
individual is identified, notify appropriate law enforcement agencies, prevent the individual 
from boarding an aircraft, or take other appropriate action with respect to that individual. 

24 Pub. L. No. 107-71, § 136, 115 Stat. 597, 637 (2001) (codified at 49 U.S.C. § 44903Q)(2)(A)) 
(requiring use of the Computer Assisted Passenger Prescreening System or any successor 
system). 
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and oversee air carrier operation of the Computer Assisted Passenger 
Prescreening System (CAPPS) — an electronic application that selects 
individuals for enhanced screening at the passenger checkpoint based on 
certain travel characteristics identified by TSA as indicating potential 
risk — and by issuing security directives in April 2002 that continued and 
amended the requirements that domestic air carriers match passenger 
information against the No Fly and Selectee lists. These security directives 
are the No Fly List Procedures security directive, requiring domestic air 
carriers to conduct checks of passenger information against the No Fly 
List to identify individuals who should be precluded from boarding flights, 
and the Selectee List Procedures security directive, directing domestic air 
carriers to conduct checks of passenger information against the Selectee 
List to identify individuals who should receive enhanced screening (e.g., 
additional physical screening or a hand-search of carry-on baggage) before 
proceeding through the security checkpoint. 25 Since 2002, TSA has issued 
numerous revisions to the No Fly and Selectee list security directives to 
strengthen and clarify requirements, and has issued guidance to assist air 
carriers in implementing their watch-list-matching processes. 26 

So that they may carry out watch-list-matching requirements, TSA 
provides air carriers with access to the No Fly and Selectee lists — subsets 
of the terrorist screening database managed by the FBI's Terrorist 
Screening Center. The terrorist screening database is composed of records 
that contain identifying information (e.g., name and date of birth) on both 
foreign and U.S. citizens with known or appropriately suspected links to 
terrorism. Only those nominations in the terrorist screening database 
submitted by elements within the intelligence community, including the 



J For the purposes of this report, we address policies and procedures applicable to air 
carriers regulated under 49 C.F.R. part 1544 (U.S.-flagged air carriers), which we refer to as 
domestic air carriers. For these air carriers, we limit our discussion to the watch-list 
matching TSA requires to secure the aviation sector for domestic flights — air carrier 
operations between two points within the United States or its territories. TSA requirements 
also address the international operations of domestic air carriers, and the operations of 
foreign-flagged air carriers flying to and from destinations within the United States and its 
territories in accordance with 49 U.S.C. part 1546; however, these requirements are outside 
the scope of our review. 

26 The most recent version of the No Fly List Procedures security directive is SD 1544-01- 
20F, dated April 9, 2008, and the most recent version of the Selectee List Procedures 
security directive is SD 1544-01-21F, dated March 8, 2007. 
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FBI, that meet criteria specified by the Homeland Security Council 27 
relating to the threat that an individual poses to civil aviation are exported 
as records to be included on the No Fly or Selectee lists. 28 At present, the 
Terrorist Screening Center forwards the No Fly and Selectee lists to TSA's 
Office of Intelligence, which generally posts new lists daily to a secure 
Web board that air carriers may access to retrieve the lists. 29 The Terrorist 
Screening Center provides TSA's Office of Intelligence with new No Fly 
and Selectee lists on a daily basis as well as any time a nominating entity 
submits additions and deletions that require immediate notification to the 
aviation community. 



TSA's Regulatory TSA is responsible for ensuring air carriers' compliance with regulatory 

Inspection Framework requirements, including requirements reflected in TSA security directives 

and TSA-approved security programs. According to TSA inspection 
guidance, compliance with regulatory requirements may be validated in 
various ways, depending on the risk associated with the requirements. For 
example, when regulatory requirements are largely administrative and 
encompass the least critical layers of security, compliance may be 
validated largely through inspections based on documentation reviews. 
However, when regulatory requirements encompass more critical layers of 
security, more intensive oversight is needed, and compliance typically is to 
be validated through testing, inspections, surveillance, special emphasis 
assessments, and special emphasis inspections. 

TSA conducts inspections of air carriers throughout the year as part of 
regular inspection cycles based on annual inspection plans. These 
inspections are based on inspection guidelines known as PARIS prompts, 



On June 10, 2008, the Department of Justice provided us comments on a draft of the 
restricted version of this report (GAO-08-453SU) and noted that the Principals Committee, 
which is a senior interagency forum under the Homeland Security Council, had approved 
additional criteria that the Terrorist Screening Center would begin implementing on June 
23, 2008. The Homeland Security Council was established to ensure coordination of all 
homeland-security-related activities among executive departments and agencies and 
promote the effective development and implementation of all homeland security policies. 
See The White House, Homeland Security Presidential Directive/HSPD-1, Organization 
and Operation of the Homeland Security Council (Washington, D.C.: Oct. 29, 2001). 

28 Each watch-list record, however, does not necessarily indicate a separate individual on 
the list. Some listed individuals have multiple records attributed to them due to the 
inclusion of known aliases and name variations. 

29 The lists may also be provided via password-protected e-mail. 
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which address a broad range of regulatory requirements (including airport 
perimeter security and cargo security, as well as screening of employees, 
baggage, and passengers). With respect to watch-list matching, an 
inspection guideline (PARIS prompt) instructs inspectors to determine, for 
example, whether the air carrier is comparing the names of all passengers 
against names on the most current No Fly and Selectee lists in accordance 
with the procedures outlined in TSA's security directives. 

TSA conducts watch-list-related inspections at air carriers' corporate 
security offices (where policies and procedures are established on how 
watch-list matching is to be performed) and at airports (where policies 
and procedures for responding to a potential match are implemented). 
TSA's principal security inspectors are responsible for conducting 
inspections at domestic air carriers' corporate headquarters. These 
inspectors assess air carriers' compliance with security requirements and 
provide direct oversight of air carriers' implementation of and compliance 
with TSA-approved security programs. TSA considers principal security 
inspectors to be subject-matter experts for the air carrier community 
concerning implementation of and compliance with security programs and 
other requirements. As of January 2008, nine principal security inspectors 
were responsible for assessing the compliance of domestic air carriers 
with requirements in the No Fly and Selectee list security directives (as 
well as with other regulatory requirements pertaining to commercial 
aviation). Each of these inspectors has responsibility for one or more 
domestic air carriers. For fiscal year 2007, there were 72 domestic air 
carriers to which the No Fly and Selectee list security directives applied. 

Field inspectors — known as transportation security inspectors — conduct 
watch-list-related inspections at airports. They are responsible for a 
multitude of TSA-related activities, including conducting inspections and 
investigations of airports and air carriers, monitoring compliance with 
applicable civil aviation security policies and regulations, resolving routine 
situations that may be encountered in the assessment of airport security, 
participating in testing of security systems in connection with compliance 
inspections, identifying when enforcement actions should be initiated, 
and providing input on the type of action and level of penalty 
commensurate with the nature and severity of a violation that is ultimately 
recommended to TSA's Office of Chief Counsel. As of June 2008, there 
were 681 transportation security inspectors responsible for 459 
commercial airports across the United States. 
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TSA began developing a program to take over watch-list-matching 
capability from air carriers in March 2003. 30 TSA cancelled this earlier 
effort, known as CAPPS II, due to development challenges and privacy 
concerns. In July 2004, the National Commission on Terrorist Attacks 
Upon the United States (the 9/11 Commission) recommended that the 
federal government take over the watch-list-matching function from air 
carriers. 31 Subsequently, the Intelligence Reform and Terrorism Prevention 
Act of 2004 required that TSA develop such a watch-list-matching 
capability. 32 Shortly after suspending work on the CAPPS II program in 
August 2004, TSA initiated development of Secure Flight, a program that 
the agency expects will allow the federal government to perform watch-list 
matching for passengers on all flights within the United States and 
ultimately for international flights with departures from or arrivals in the 
United States. 

In February 2006, we testified that although some progress had been made 
in developing Secure Flight, long-standing issues related to systems 
development and testing, program management, privacy protections, and 
redress remained. 33 We reported in testimony that as a result of these 
deficiencies the program was at risk of failure. Following our February 
2006 testimony, TSA announced a temporary suspension of Secure Flight's 
development to reassess program goals and capabilities. TSA completed 
this reassessment in January 2007, moved forward to complete its 



TSA initiated this effort in response to the Aviation and Transportation Security Act, 
which requires that TSA ensure that a system is used to evaluate all passengers before they 
board an aircraft and ensure that selected individuals and their carry-on and checked 
baggage are adequately screened. See Pub. L. No. 107-71, § 136, 115 Stat, at 637 (codified at 
49 U.S.C. § 44903Q)(2)(A)). 

31 The National Commission on Terrorist Attacks Upon the United States, The 9/11 
Commission Report - Final Report of the National Commission on Terrorist Attacks 
Upon the United States (Washington, D.C.: 2004), p. 393. 

32 Pub. L. No. 108-458, § 4012(a)(1), 118 Stat. 3638, 3714-17 (2004) (codified at 49 U.S.C. 

§ 44903(j)(2)(C) (2004)). A separate provision enacted at section 4012(a)(2) addressed the 
predeparture screening of international passengers, with the Secretary of Homeland 
Security giving this responsibility to U.S. Customs and Border Protection. See 49 U.S.C. 
§ 44909(c)(6). 

33 With regard to redress protections, DHS must have a process whereby aviation 
passengers determined to pose a threat to aviation security by Secure Flight may appeal 
that determination and correct erroneous information contained within the prescreening 
system. See GAO, Aviation Security: Significant Management Challenges May Adversely 
Affect Implementation of the Transportation Security Administration's Secure Flight 
Program, GAO-06-374T (Washington, D.C.: Feb. 9, 2006). 



Secure Flight: 
Development of a 
Government-Run Watch- 
List-Matching Process 
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concept-of-operations plan for the Secure Flight program and strengthen 
systems development efforts, and, in August 2007, issued a notice of 
proposed rulemaking describing the requirements TSA will expect air 
carriers to implement to facilitate the government-run prescreening 
process. 34 TSA expects that, beginning in early calendar year 2009, the 
Secure Flight program will begin assuming from air carriers the watch-list- 
matching responsibility for domestic flights. At some point following this 
assumption for domestic flights, TSA plans to assume from U.S. Customs 
and Border Protection this watch-list-matching function for international 
flights that depart from or arrive in the United States. However, we 
testified in February 2008 that despite significant progress in the 
development of Secure Flight, TSA did not fully follow best practices for 
developing Secure Flight's life-cycle cost and schedule estimates, and that 
failure to do so put the program at risk of cost overruns, missed deadlines, 
and performance shortfalls, among other issues. 35 



TSA Took Action in 
2008 to Enhance 
Watch-List Matching 
Conducted by Air 
Carriers but Believes 
the Ultimate Solution 
Will Be 

Implementation of 
Secure Flight 



Through its security directives, TSA has issued requirements for watch-list 
matching, which include identifying passengers with names similar to 
those on the No Fly and Selectee lists — a process TSA refers to as similar- 
name matching. Before undertaking revisions of the relevant security 
directives in 2008, TSA expected air carriers to conduct similar-name 
matching but TSA's security directives did not specify how many and what 
types of such name variations air carriers should compare. Consequently, 
some of the 14 air carriers we interviewed reported that they compared 
more name variations than others. Air carriers that do not conduct similar- 
name comparisons and carriers that conduct relatively limited 
comparisons are less effective in identifying watch-listed individuals who 
travel under name variations. Also, due to inconsistent air carrier 
processes, a passenger could be identified as a match by one carrier and 
not by another. In April 2008, during the course of our review, TSA revised 
and issued the No Fly List security directive to specify a baseline 
capability for similar-name matching to which all air carriers must 
conform. Also, in April 2008, TSA officials reported that the agency had 



See 72 Fed. Reg. 48,356 (Aug. 23, 2007). Requirements described in the notice of 
proposed rulemaking are subject to revisions based on various considerations, including 
input that TSA received during the public comment period. As of the date of this report's 
issuance, DHS had not issued a final Secure Flight rule. 

35 GAO, Aviation Security: Transportation Security Administration Has Strengthened 
Planning to Guide Investments in Key Aviations Security Programs, but More Work 
Remains, GAO-08-465T (Washington, D.C.: Feb. 28, 2008). 
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plans to similarly revise the Selectee List security directive to require the 
same baseline capability. 36 TSA officials acknowledged that the new 
baseline capability will not address all vulnerabilities identified by TSA. 
However, the officials stated that the new baseline capability was their 
best interim approach for improving air carriers' matching efforts because, 
among other reasons, it will strengthen watch-list matching without 
requiring considerable investment in a solution that will be replaced when 
Secure Flight is implemented. TSA officials further stated that the longer 
term solution for watch-list matching is Secure Flight, which will have the 
capability to undertake more advanced searches for individuals on the No 
Fly and Selectee lists. 



Prior to April 2008, TSA 
Watch-List-Matching 
Requirements Were Broad 
and Allowed Air Carriers 
to Implement Less 
Effective Processes 



Prior to a revision of the No Fly List security directive in April 2008 — and a 
similar revision planned for the Selectee List security directive — TSA's 
watch-list-matching requirements for domestic flights (summarized in 
table 1) addressed five key processes: (1) retrieval of the No Fly and 
Selectee lists, (2) the matching of passenger and list information, (3) the 
use of TSA's Cleared List, 37 (4) notification procedures, and (5) record- 
keeping activities. 38 In April 2008, TSA revised the No Fly List security 
directive for watch-list matching and also reported plans for similarly 
revising the Selectee List security directive. The security directive 
revisions — discussed later in this section — still address the five key 
process areas, but provide greater specificity on TSA's requirements for 
matching passenger and watch-list information (the second key process 



In September 2008, TSA informed us that the revised Selectee List security directive was 
still in the agency's internal clearance process but did not provide us a targeted issuance 
date. 

37 When making determinations on matches, air carriers must use the TSA Cleared List, 
which is composed of names and other personal-identifying information on individuals 
whom the Department of Homeland Security has reviewed and determined are not 
individuals on the No Fly or Selectee lists. 

38 Specifically, we reviewed and discussed the No Fly and Selectee list security directives 
and identified within each the key requirements pertaining to domestic flights. Although 
the same requirements generally apply to the international flights of both domestic and 
foreign air carriers, such operations fall outside the scope of our review. For more 
information on how we identified requirements for watch-list matching, see appendix I. 
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shown in table l). 39 Prior to the April 2008 revision of the No Fly List 
security directive, TSA's requirements in this area lacked specificity for 
purposes of implementation, although the then-current security directives 
addressed the need for air carriers to identify passengers with names that 
are either identical or similar to those on the No Fly List or the Selectee 
List. To identify passengers with similar names — an activity known as 
similar-name matching — air carriers' automated programs or manual 
reviews were expected to capture No Fly and Selectee list names that are 
variations of the name on the passenger's reservation. 



TSA's revised No Fly List Procedures security directive (SD 1544-01-20F) is dated April 9, 
2008. Also, in April 2008, TSA reported that the current Selectee List Procedures security 
directive (SD 1544-01-21F) would be similarly revised. In September 2008, TSA informed us 
that the revised Selectee List security directive was still in the agency's internal clearance 
process but did not provide us a targeted issuance date. 
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Table 1 : TSA Watch-List-Matching Requirements Prior to the April 2008 Revision to the No Fly List Security Directive 



Requirements (key processes) Discussion 

(1 ) Retrieving the No Fly and Selectee lists • Air carriers must monitor the TSA Web board throughout the day for the most recent 

postings of the No Fly and Selectee lists. 

Within 24 hours of scheduled flight departure time, but no later than passenger check- 
in, air carriers are to compare records from the most recently issued No Fly and 
Selectee lists with identifying information on passengers found in the respective air 
carrier's reservation system and offered by passengers at the time of check-in. 

When comparing data, air carriers must identify name matches to the No Fly and 
Selectee lists. To identify similar-name matches, automated and manual processes 
are expected to have the capability to compare name variations. 

To determine which passengers are matches, a passenger's name and one piece of 
identifying information (found either within the air carrier's reservation system or 
supplied by the passenger at check-in) must match with corresponding information 
provided on the No Fly or Selectee lists. 

(3) Using the TSA Cleared Lisf* • When making determinations on matches, air carriers must use the TSA Cleared List, 

which is composed of names and other personal-identifying information on individuals 
whom the Department of Homeland Security has reviewed and determined are not 
individuals on the No Fly or Selectee lists. Individuals determined to be on the TSA 
Cleared List should be accepted for travel and not be subject to further procedures for 
handling matches to No Fly or Selectee lists identified in the security directives. 

(4) Notifying authorities • Upon identifying a passenger whose information matches with the No Fly or Selectee 

lists and who is not on the TSA Cleared List, air carriers must follow certain 
notification procedures, such as to contact the federal security director and the 
appropriate local law enforcement officer (for matches to the No Fly List) or to 
designate the passenger as a selectee for enhanced checkpoint screening 
procedures (for matches to the Selectee List). 

(5) Keeping records • Air carriers must keep records on the results of watch-list matching for specified time 

periods — for example, air carriers must keep a record of all flights operated with 
passengers designated as selectees for 7 calendar days from the date of the flight's 
departure. 



Sources: GAO analysis of TSA's No Fly List Procedures security directive (SD 1 544-01 -20 series) and Selectee List Procedures 
security directive (SD 1544-01-21 series), versions dated July 8, 2004, and March 8, 2007. 

"Security directives in effect prior to the April 2008 revision of the No Fly List Procedures security 
directive referenced a "cleared column," a format for clearing passengers. TSA eventually replaced 
this format with the Cleared List, and revised language for the April 2008 No Fly List security 
directive. 

Air carriers must conduct similar-name matching because watch-listed 
individuals may travel using variations of the names attributed to them on 
the No Fly or Selectee lists and, thus, would not be identified if air carriers 
searched only for an exact-name match. At present, TSA does not require 
that air carriers collect the full name from passengers making travel 
reservations, thus, passengers may travel using variations of their legally 
documented names; for example, abbreviated name forms or portions of 
their names. Such name variations may arise due to unintentional 
errors — for example, a travel agent mistakenly books travel for "Jon" 



(2) Matching passenger data to No Fly and • 
Selectee lists 
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when the name spelling is actually "John," or the agent accidentally 
transposes a passenger's first and middle names for a flight reservation. 
Traveling under a name variation could also represent a watch-listed 
individual's intentional effort to evade detection. For example, an 
individual identified as John Robert Smith on his driver's license may 
make a travel reservation using a common name variation — such as using 
his middle and last names (Robert Smith) or his initials and last name (J.R. 
Smith). If the John Robert Smith in this example were a name on the No 
Fly List, an exact, letter-for-letter comparison of the passenger's 
reservation name (either Robert Smith or J.R. Smith) with the No Fly List 
would fail to identify the watch-listed individual. However, a comparison 
of possible variations of the watch-list name (John Robert Smith) could 
identify either Robert Smith or J.R. Smith as a potential match — that is, an 
individual who is a possible match to the No Fly List or Selectee List and 
whose personal identifying information requires further review before a 
match can be determined. 

Before 2008, TSA's Security Regarding similar-name matching, before 2008, TSA's security directives 

Directives Allowed Air Carriers na cl broad requirements that allowed air carriers discretion in determining 
More Discretion in Comparing the extent to which they compared name variations. For instance, to 
Name Variations identify watch-listed individuals who travel using variations of their name, 

TSA's security directives did not specify how many possible combinations 
of name elements should be compared. TSA officials explained that the 
agency initially issued broad security directives to allow air carriers 
flexibility in implementing requirements and — until the April 2008 revision 
of the No Fly List security directive — left the directives relatively 
unchanged because the agency was developing a government-run 
capability to take over this function. The operations of those air carriers 
that are subject to the watch-list-matching requirements of TSA's security 
directives range from commuter providers to international-service 
providers. According to TSA officials, broad security directive 
requirements permit air carriers with such diverse operations to 
implement processes that best meet their operational needs and 
technological capabilities. 

Officials further explained that TSA's focus has been on developing its 
own watch-list-matching capability (now Secure Flight) since 2003. TSA 
officials noted that, though not an impetus for making requirements broad 
when first articulated in 2002, this focus on developing a government-run 
watch-list-matching program is one reason why these requirements 
remained relatively unchanged until April 2008. 
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Failure to Conduct Similar- 
Name Matching or Comparing 
Name Variations to a Lesser 
Extent Reduces the 
Effectiveness of Watch-List 
Matching 



The 14 air carriers we interviewed reported adopting different approaches 
to name matching. Although each of the 14 air carriers we spoke with 
during our review reported conducting comparisons to identify exact- 
name matches of passengers and names on the No Fly List or the Selectee 
List, not every air carrier reported conducting similar-name comparisons. 40 
Those air carriers that conducted similar-name comparisons reported 
using various approaches, some of which compared more name variations 
than others. 



According to air carriers, a critical factor affecting their implementation of 
similar-name-matching requirements was their observation that 
conducting more comparisons for variations results in longer lines at 
ticket counters and passenger inconvenience. Specifically, 10 air carriers 
commented that conducting similar-name comparisons resulted in more 
passengers being identified as potential matches. At the time of check-in, 
air carriers must perform additional checks at the ticket counter of each 
potentially matched passenger's government-issued identification against 
data on the No Fly and Selectee lists. Therefore, according to 12 of the 14 
air carriers we spoke with, a large number of potential matches can lead to 
congestion at the ticket counter and longer wait times for all passengers. 

Inconsistent approaches to conducting similar-name matching could lead 
a passenger to be identified as a match by one air carrier and not by 
another. Further, not conducting similar-name matching — or conducting 
such matching to only a very limited extent — compromises the usefulness 
of the No Fly List and Selectee List. There have been incidents, according 
to TSA's Office of Intelligence, of air carriers failing to identify potential 
matches by not effectively conducting similar-name matching. In these 
incidents, the air carriers' processes led to false negative watch-list- 
matching results — that is, individuals who were on the No Fly List and 
were not identified by the respective air carrier's watch-list-matching 
process. In some of these incidents, the individual's flight reservation 
contained a name that varied somewhat from the name on the No Fly List, 
and the air carrier's watch-list-matching process did not identify the name 
as a possible match. 



We did not independently verify the air carriers' approaches to watch-list matching. 
Unless noted otherwise, our summary of the air carriers' approaches is based on system 
capabilities reported to us in 14 separate interviews with the respective air carriers. 
Appendix II provides more detail on the 14 air carriers' reported approaches to watch-list 
matching. 
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In most of these cases, the failures of the air carriers to identify the 
potential matches were discovered as a result of the U.S. Customs and 
Border Protection's comparison of passenger and watch-list data for 
international flights. Specifically, TSA learned of the failures through U.S. 
Customs and Border Protection, which identified the No Fly listed 
individual when conducting its own comparison of passenger information 
against the No Fly and Selectee lists for international flights. 41 These 
comparisons, performed as part of U.S. Customs and Border Protection's 
border security mission, took place after the air carriers completed their 
comparisons, in effect constituting a second check of passenger and 
watch-list information. U.S. Customs and Border Protection does not 
screen passengers on domestic flights; thus, there is no opportunity for a 
second comparison of passenger information against the No Fly and 
Selectee lists for domestic flights. Therefore, it is difficult to determine the 
extent to which domestic air carriers may be failing to identify watch- 
listed individuals who are able to board domestic flights. 

In October 2007, we reported that of the known cases in which individuals 
on the No Fly List flew on international flights bound to or from the United 
States, some were allowed to fly because the respective air carrier's 
process failed to identify the passenger's name as a match. 42 Although 
these individuals were subsequently identified in-flight by other means, the 
onboard security threats required an immediate counterterrorism 
response, which in some instances resulted in diverting the aircraft to a 
location other than its original destination. 43 According to TSA's Office of 
Intelligence, some of these incidents may be attributed to air carriers' 
inability to identify similar-name matches when passengers travel using 
variations of their name. 

TSA had been aware that air carriers were not using equivalent processes 
to compare passenger names with names on the No Fly and Selectee lists. 
For instance, in June 2006, we reported that the improvements air carriers 



Some of these flights involved passengers who flew from one domestic location to 
another domestic location, where they boarded an international flight. TSA learned that the 
individual on the No Fly List flew domestically after U.S. Customs and Border Protection 
identified the individual on the international leg. 

42 GAO, Terrorist Watch List Screening: Opportunities Exist to Enhance Management 
Oversight, Reduce Vulnerabilities in Agency Screening Processes, and Expand the Use of 
the List, GAO-08-110 (Washington, D.C.: Oct. 11, 2007). 

43 GAO-08-110. 
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were making to their individual watch-list-matching processes, though 
beneficial to the respective air carrier's operations, could further 
exacerbate differences that currently exist among the various air carriers 
and could result in varying levels of effectiveness across air carriers in 
matching passenger information to the No Fly and Selectee lists. 44 
Furthermore, TSA's March 2007 Secure Flight Program Baseline explained 
"because each aircraft operator conducts its own matching process, the 
ability to conduct watch-list matching and coordinate law enforcement 
responses is not consistent across the aviation industry." 45 Moreover, in 
several interviews over the course of our work, TSA officials 
acknowledged that in general, some air carriers were performing more 
similar-name comparisons than other air carriers. TSA's understanding of 
the significance of these differences was crystallized in January 2008, 
when results of a special emphasis inspection identified deficiencies in air 
carriers' similar-name-matching capability. 



To Address Deficiencies in 
Air Carriers' Similar-Name- 
Matching Capability, TSA 
Issued a Revised No Fly 
List Security Directive in 
April 2008 to Provide More 
Specific Requirements 



During the course of our work and in response to findings of the January 
2008 special emphasis inspection that identified deficiencies in air carriers' 
similar-name-matching capability, TSA officials reported that the agency 
immediately began to assess options for corrective actions to implement 
across the aviation industry. In doing so, officials noted that they 
consulted with representatives from the intelligence community, the 
Secure Flight program, and the aviation industry. On the basis of its 
assessment, TSA revised the No Fly List security directive in April 2008 to 
establish a specific baseline capability for air carriers in conducting 
similar-name matching. Also, in April 2008, TSA officials reported that the 
agency had plans to similarly revise the Selectee List security directive to 
require the same baseline capability. 46 



GAO, Aviation Security: Management Challenges Remain for the Transportation 
Security Administration's Secure Flight Program, GAO-06-864T (Washington, D.C.: June 
14,2006). 

45 Upon completing a reassessment of the Secure Flight program in February 2007, TSA 
produced this document to identify decisions made about Secure Flight's capabilities 
during the reassessment. See TSA, Secure Flight Program Baseline (Washington, D.C.: 
March 2007), p. 5. 

46 As mentioned previously, in September 2008, TSA informed us that the revised Selectee 
List security directive was still in the agency's internal clearance process but did not 
provide us a targeted issuance date. 
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TSA officials acknowledged that the new baseline capability will not 
address all vulnerabilities identified by TSA. However, TSA officials 
explained that they expect the new similar-name matching baseline 
capability to strengthen the watch-list matching currently performed by air 
carriers. In particular, the officials expect the newly established baseline 
capability to improve the matching processes of those air carriers that do 
not compare the kinds of variations required by the new baseline or that 
compare none at all. Furthermore, according to agency officials, the 
variations specified by the new baseline address the types of situations air 
carriers will encounter due to passengers making their own reservations. 
Accordingly, TSA concluded that requiring air carriers to conduct similar- 
name comparisons beyond the baseline capability specified in the revised 
No Fly List security directive was not warranted for the interim period 
pending the implementation of Secure Flight. TSA was not able to provide 
us with data or analysis to support this assertion, and we did not 
undertake an independent analysis to determine the sufficiency of the 
newly established baseline. 

TSA officials also explained they determined that revising the security 
directives to be the most feasible approach for strengthening the current 
watch-list-matching process over other options because it was expedient 
and would have the least negative impact on air carriers' operations. 
Specifically, TSA officials determined that upon issuing the revised No Fly 
List security directive, air carriers would need only 2 to 4 weeks to 
implement new requirements. When considering how this option would 
affect air carrier operations, TSA officials explained that they considered 
the number of potential matches that likely would be generated by the 
new baseline capability. As previously discussed, air carriers reported that 
comparing more name variations results in more passengers being 
identified as potential matches, who then must go to the ticket counter to 
obtain their boarding passes. Thus, large numbers of potential matches 
could overwhelm air carriers' check-in operations. TSA officials explained 
that the industry officials with whom they consulted in developing the new 
baseline capability believed it would produce a manageable number of 
potential matches. 

In exploring actions to strengthen the watch-list-matching process, TSA 
considered two other options — one that would have required each air 
carrier to contract with third-party providers to develop customized 
watch-list-matching software, and another that involved the creation of an 
expanded version of the No Fly and Selectee lists to include name 
variations so that air carriers need only conduct comparisons to identify 
an identical match. TSA identified significant obstacles to implementing 
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these options. Specifically, TSA determined that contracting with third- 
party vendors was impracticable due to availability and timing concerns. 
For instance, identifying appropriate vendors and implementing vendor- 
provided solutions could take almost 2 years — an unrealistic time frame 
given that Secure Flight's implementation is scheduled to begin in 2009. In 
this regard, TSA officials also expressed reluctance to requiring air 
carriers to undertake the expense of contracting with third-party vendors 
for an interim approach, while at the same time requiring that air carriers 
invest in system changes for Secure Flight. With regard to the option of 
adding name variations to the No Fly and Selectee lists, according to TSA 
officials, creating these variations would have greatly expanded the total 
size of the No Fly List, which could overwhelm the name-matching 
capability of some air carriers and could potentially send an 
unmanageable number of potential matches to the ticket counters of air 
carriers. As previously discussed, in our air carrier interviews, 10 of the 14 
air carriers reported that searching for more name variations leads to the 
identification of more potential matches. In this regard, there is some 
support for TSA's determination that expansion of the No Fly and Selectee 
lists could produce an unmanageable number of potential matches. 
However, we did not independently assess this issue. 

Although TSA officials characterized the new baseline capability as a good 
interim solution for strengthening watch-list matching — one that balances 
TSA's need to strengthen watch-list matching with the air carriers' need 
for efficient operations — they stressed that the Secure Flight program is 
ultimately the solution. For example, in its development of Secure Flight, 
TSA plans to develop a name-matching process that will have the 
capability to identify name variations beyond those specified by the new 
baseline. Further, according to TSA, Secure Flight will be better able to 
use passenger names and other identifying information (such as date of 
birth and gender) to more accurately match passengers to the subjects of 
watch-list records and, thereby, further reduce the risks of false negatives 
without unacceptably increasing the number of false positives (mistakenly 
identifying a passenger's name as a potential match with watch-list 
records). 
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Until a 2008 Special 
Emphasis Inspection, 
TSA Had Conducted 
Limited Testing of Air 
Carriers' Capability to 
Perform Similar-Name 
Matching 



Although TSA assessed air carriers' compliance with watch-list-matching 
requirements through a special emphasis assessment conducted in 2005 
and through planned inspections conducted in conjunction with annual 
inspection cycles, the agency had tested similar-name matching to a 
limited extent until 2008. For instance, the 2005 special emphasis 
assessment focused on air carriers' capability to identify passenger names 
that were exact matches with names on the No Fly List, but did not 
address the capability to conduct similar-name matching. Also, during the 
most recent annual inspection cycle (fiscal year 2007), although some TSA 
inspectors tested air carriers' effectiveness in conducting similar-name 
matching, the inspectors did so at their own discretion and without 
specific evaluation criteria. However, during a special emphasis inspection 
conducted in January 2008, TSA found deficiencies in the capability of air 
carriers to conduct similar-name matching. 47 Thereafter, following TSA's 
revision of the No Fly List security directive in April 2008, officials planned 
to issue new guidance for inspectors to better ensure compliance by air 
carriers with requirements in the new security directive (e.g., by providing 
uniform evaluation criteria consistent with the new requirements). In 
response to our request for updated information on its oversight efforts, 
TSA provided us the results of a special emphasis assessment (conducted 
in May 2008) of seven air carriers' compliance with the revised No Fly List 
security directive. Although the details of this special emphasis 
assessment are classified, TSA officials generally characterized the results 
as positive. Further, TSA's noted that the agency's internal handbook — 
which provides guidance to transportation security inspectors on how to 
inspect air carriers' performance of various requirements, including watch- 
list-matching requirements — was being revised and was expected to be 
released later this year. Thus, TSA indicated that the new inspection 
guidance would be used in conjunction with the nationwide regulatory 
activities plan for fiscal year 2009. While these actions and plans are 
positive developments, it is too soon to determine the extent to which air 
carriers' compliance with watch-list-matching requirements will be 
assessed based on the new security directives since these efforts are still 
underway. 



TSA reported that the January 2008 special emphasis inspection covered 52 domestic air 
carriers and 31 foreign air carriers. 
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TSA's Special Emphasis 
Assessment in 2005 
Focused on Air Carriers' 
Exact-Name-Matching 
Capability 



TSA conducted a special emphasis assessment in 2005 that tested the 
capability of domestic air carriers to find passenger names that were exact 
matches to names on the No Fly List. The 2005 special emphasis 
assessment was undertaken at the request of the TSA Administrator due to 
serious failures in air carriers' watch-list-matching processes, according to 
a senior TSA official. To conduct the assessment, TSA inspectors made 
flight reservations using the exact name of an individual who was on the 
No Fly List and not on the TSA Cleared List. If the air carrier identified the 
name on the reservation as a potential match to the individual on the No 
Fly List — and the check-in agent identified through the reservation system 
that further assistance was needed to finish the check-in process (e.g., to 
call security) — the test was considered to be successfully completed. 
According to TSA data: 



air carriers passed a large majority of the initial tests conducted in 
June and July 2005, although several air carriers failed one or more 
tests and 



those air carriers that failed a test were retested in September 2005, 
and a large majority of these air carriers passed the tests. 48 



Although TSA conducted a large number of tests, TSA officials stated — 
and our own analyses confirmed — that results from this special emphasis 
assessment would not produce a reliable estimate of the success rate of all 
attempted matches by air carriers because TSA did not randomly select 
the air carriers, airports, or individual flights for review. As a result, the 
findings from this assessment cannot be used to infer overall or individual 
rates of success in identifying exact name matches in accordance with the 
No Fly and Selectee list security directives. That is, although the 2005 
special emphasis assessment provided insight into air carriers' 
effectiveness in conducting a basic form of name matching, the picture 
provided was incomplete. Moreover, the air carriers' failure rates may 
have been considerably higher had the special emphasis assessment tested 
similar-name-matching capability, given that this capability involves more 



According to TSA officials, the agency had planned to conduct tests of all 81 domestic air 
carriers that were subject to the No Fly List Procedures security directive at that time. 
However, the officials explained that due to limited resources, initial testing covered 63 air 
carriers (encompassing operations at 354 airports), and the retesting covered 36 air carriers 
(encompassing operations at 290 airports). 
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than finding a name that is a letter-for-letter match to another name. 
However, TSA officials told us that at the time of the special emphasis 
assessment in 2005, exact-name matching was the agency's focus. 



TSA Conducted Planned 
Watch-List-Related 
Inspections throughout the 
Year, but Inspectors Tested 
Air Carriers' Effectiveness 
at Similar-Name Matching 
at Their Own Discretion 
and without Baseline 
Evaluation Criteria 



Since issuing the No Fly and Selectee list security directives in 2002, TSA 
has incorporated watch-list-related inspections into its regular inspection 
cycle, but inspectors tested air carriers' effectiveness in similar-name 
matching during these planned inspections to a limited extent and without 
specific evaluation criteria. In the most recent annual inspection cycle 
(fiscal year 2007), TSA conducted 1,145 inspections of air carriers' 
compliance with watch-list-related requirements in the No Fly and 
Selectee security directives; 1,109 of these inspections were conducted at 
air carriers' airport locations by transportation security inspectors and 36 
at air carriers' corporate security offices by principal security inspectors. 49 
The 1,145 inspections covered 60 of the 72 domestic air carriers to which 
the security directives applied during fiscal year 2007, and most of the 
carriers were inspected multiple times that year. 50 TSA found air carriers in 
compliance with required procedures in 1,133 (99 percent) of the 1,145 
inspections. 51 



These inspections were based on one or more inspection guidelines 
(called PARIS prompts) and were sometimes conducted in combination 
with inspections related to other regulatory requirements, such as 
performing criminal history record checks on employees or implementing 
CAPPS procedures. Table 2 presents the inspection guidelines TSA used to 
assess a key security directive requirement that we reviewed — matching 



As noted earlier, we concluded that these inspection data were sufficiently reliable for 
the purposes of this report, but we have concerns about the potential for error based on 
TSA's process for querying its inspection database (we discuss these concerns in more 
detail in app. I). 

50 Regarding the air carriers that did not receive a watch-list-related inspection during fiscal 
year 2007, TSA does not require inspectors to inspect each air carrier every year in terms of 
watch-list-related requirements. However, a senior TSA official in the compliance area who 
supervises inspectors stated that annually inspecting every air carrier is a goal, at least for 
principal security inspectors. 

51 We did not evaluate the basis for the inspectors' assessment decisions regarding 
compliance with requirements. Although TSA's security directives require comparisons of 
passenger and employee names to the No Fly and Selectee lists, our review was confined to 
requirements related to passengers only. 
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passenger names to the No Fly and Selectee lists. 52 Additional guidelines 
used to assess other requirements in our review are presented in appendix 

J 53 



Table 2: Requirements for Matching Passenger Data to No Fly and Selectee Lists and Inspection Guidelines Used to Assess 
Compliance with the Requirements 



Inspection guidelines 



Requirements for matching passenger 
data to No Fly and Selectee lists 

• Within 24 hours of scheduled flight 
departure time, air carriers are to 
compare records from the most recently 
issued No Fly and Selectee lists with 
identifying information on passengers 
found in the respective air carrier's 
reservation system and offered by 
passengers at the time of check-in. 

• When comparing data, air carriers must 
identify name matches (including similar- 
name matches) to the No Fly and 
Selectee lists. 

• To determine which passengers are 
matches, a passenger's name and one 
piece of identifying information (found 
either within the air carriers' reservation 
system or supplied by the passenger at 
check-in) must match with 
corresponding information provided on 
the No Fly or Selectee lists. 



Transportation security inspectors 

• All passenger names are compared to 
the most current No Fly and Selectee 
lists. 

• The aircraft operator is comparing all 
passenger names to the most current No 
Fly and Selectee lists in accordance with 
the procedures outlined in Security 
Directive 1544-01-20 series (No Fly) and 
Security Directive 1544-01-21 series 
(Selectee). 



Principal security inspectors 

• Procedures are in place to ensure the 
most recently issued No Fly List is 
utilized within 24 hours of receipt. 

• Procedures are in place to ensure the 
most recently issued Selectee List is 
utilized within 24 hours of receipt. 

• Procedures are in place to contact the 
Federal Security Director, local law 
enforcement, the FBI, and TSA Office of 
Intelligence for matches to the No Fly 
List. 

• Records are maintained of all flights 
operated with passengers who were 
determined by local law enforcement, 
U.S. legal attache, or TSA Office of 
Intelligence not to be a match. 



Sources: GAO analysis of TSA's No Fly List Procedures security directive (SD 1 544-01 -20 series) and Selectee List Procedures 
security directive (SD 1544-01-21 series), versions dated July 8, 2004, and March 8, 2007, and inspection guidelines applicable during 
fiscal year 2007. 



The inspections conducted by transportation security inspectors at 
airports used the guidelines in table 2 to assess air carriers' compliance in 
matching passenger data to the No Fly and Selectee lists in fiscal year 
2007. However, these inspectors tested exact-name and similar-name 
matching during these inspections at their own discretion; moreover, an 
official in TSA's Office of Security Operations, Compliance Division, stated 
that, generally, transportation security inspectors test exact-name- 



To report their findings in TSA's automated database, inspectors select one of four 
options from a computer-generated list: not inspected, not applicable, not in compliance, 
and in compliance. If the inspectors wish to add narrative to describe their findings, they 
can do so in a data field reserved for comments. 

53 In appendix I, see table 3. 



Page 28 



GAO-08-992 Aviation Security and Watch List Matching 



matching capability only. This inspection guideline is broadly written and 
does not specify the methods for validating compliance with the 
requirement to perform name comparisons. According to a TSA official in 
the Office of Security Operations, field inspectors may validate compliance 
by asking check-in agents to demonstrate that they have access to the 
current No Fly and Selectee lists and that any hard copies of the lists are 
properly protected; they may also interview check-in agents to ensure that 
they understand the security directive requirements, observe them as they 
process passengers who have been identified as Selectee or No Fly 
individuals, and/or test the air carriers' system by requesting a gate pass in 
the name of an individual on the watch list. We found evidence of field 
inspectors testing air carriers' name matching systems in 55 of the 1,109 
inspections they conducted in fiscal year 2007 (such tests may have been 
administered during the other inspections conducted in fiscal year 2007 
but were not documented). 

For the 36 inspections conducted by principal security inspectors at air 
carriers' corporate security offices, we found 6 inspection records that 
referred to tests of exact-name and similar-name matching capability (they 
may have administered such tests during the other inspections they 
conducted that year but did not document the tests). Principal security 
inspectors did not have an inspection guideline directing them to assess 
exact-name and similar-name matching capability specifically — thus they 
tested this capability at their own initiative, and then reported their 
methods and results in conjunction with one of the four guidelines 
presented in table 2. Further, in response to our inquiry, 6 of TSA's 9 
principal security inspectors told us that their assessments have not 
included examining air carriers' capability to conduct certain basic types 
of similar-name comparisons. 

TSA establishes in guidance for inspections (including watch-list-related 
inspections) that testing is the preferred method for assessing air carriers' 
compliance with regulations whenever possible and that it is only through 
testing that security can be assured. 54 TSA further establishes in inspection 
guidance that when regulatory requirements encompass critical layers of 
security, more intensive oversight is needed, and compliance typically is to 
be validated through testing, inspections, surveillance, special emphasis 



TSA, National Inspection Manual, 2007. Inspections for all regulated areas (not just 
watch-list-related inspections) generally incorporate all of four methods — testing, 
document review, interviews, and surveillance. 
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assessments, and special emphasis inspections. 55 Without routinely testing 
air carriers' compliance with the similar-name-matching requirement, TSA 
may not have reliable data on the effectiveness of air carriers' watch-list- 
matching processes and could be hindered in taking timely action to 
address any deficiencies. 

Inspectors who have tested air carriers' effectiveness in performing 
similar-name matching have done so without specific evaluation criteria. 
As discussed earlier, for any given name there are a number of possible 
name variations that could be used for travel, but TSA inspectors did not 
have baseline criteria on the number or types of such variations that must 
be evaluated. In the absence of specific standards for similar-name 
matching that all air carriers must follow, TSA has had no assurance that 
its inspections are based on uniform evaluation criteria. The inspections 
may not have been conducted uniformly and may have produced 
inconsistent results, given the absence of specific standards. In fall 2007, 
TSA began to review the adequacy of inspection guidance used by 
principal security inspectors, including guidance for watch-list-related 
inspections. As discussed in the following section, TSA expects to provide 
baseline criteria on the number and types of such variations inspectors 
must evaluate, but had not completed these efforts as of early September 
2008. 



A Special Emphasis 
Inspection Conducted in 
2008 Found Deficiencies in 
Air Carriers' Similar-Name- 
Matching Capabilities, and 
TSA Has Plans for 
Corrective Actions 



During the course of our review and following TSA's discovery of a major 
air carrier's inability to effectively conduct both exact-name and similar- 
name-matching against the No Fly List, TSA initiated a 3-day, special 
emphasis inspection in January 2008 that tested the capability of 83 air 
carriers to conduct watch-list matching. 56 According to TSA officials, this 
inspection covered 52 domestic air carriers and 31 foreign air carriers. To 
implement the special emphasis inspection, TSA used 100 names on the 
No Fly List to test the 83 air carriers' capability to identify both exact- 
name and similar-name matches based on various types of possible name 
variations. On the basis of test results, a senior TSA official stated that the 
agency has confidence in air carriers' capability to identify exact-name 
matches. Regarding the capability to identify similar-name matches, TSA 



J TSA, Regulatory Activities Plan for Transportation Security Inspectors Fiscal Year 
2008. 

56 We briefed the TSA Administrator and other senior officials on the results of our work in 
November 2007. 
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found that no air carrier was successful in identifying matches involving 
all types of name variations, although some carriers were more effective 
than others. 

On the basis of this inspection, TSA officials stated that they began to 
strengthen oversight of air carriers' similar-name-matching capability. For 
example, the TSA officials explained that — after a 30-day period following 
issuance of the revised No Fly List security directive in April 2008 — the 
agency's inspectors would begin to evaluate air carriers' performance in 
complying with the new requirements. TSA officials explained that these 
initial inspections would be conducted at air carriers' corporate security 
offices and at airports. Officials further stated that after these initial 
inspections, others would be conducted periodically and, if applicable, 
TSA would impose progressively stronger enforcement actions against air 
carriers that are not successful in meeting the new standards. 

In September 2008, in response to our request for updated information on 
the status of its oversight efforts, TSA provided us the results of a special 
emphasis assessment (conducted during May 20-29, 2008) of seven air 
carriers' compliance with new requirements in the No Fly List security 
directive. Although the details of this special emphasis assessment are 
classified, TSA generally characterized the results as positive. Also, TSA 
plans to work with individual air carriers, as applicable, to analyze specific 
failures, improve system performance, and conduct follow-up testing as 
needed. 

In further reference to revision of the No Fly List security directive in April 
2008, TSA officials stated that the agency's internal guidance is being 
updated to align inspection guidance with the revised directive. The 
officials elaborated that the new inspection guidance will place more 
emphasis on testing the effectiveness of security measures rather than 
using a checklist approach to determine whether an air carrier has a 
particular procedure in place. Regarding the emphasis on testing, our 
review noted that the draft guidance being developed for principal security 
inspectors included testing scenarios based on the types of name 
variations that air carriers must be capable of conducting in accordance 
with the revised watch-list-matching requirements. Also, according to TSA, 
guidance for transportation security inspectors is being developed (as part 
of the 2009 Regulatory Activities Plan) to provide more specific direction 
to inspectors for assessing name-matching capability. In September 2008, 
in response to our inquiry, TSA noted that the agency's internal 
handbook — which provides guidance to transportation security inspectors 
on how to inspect air carriers' performance of various requirements, 
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including watch-list-matching requirements — was being revised and was 
expected to be released later this year. Thus, TSA indicated that the new 
inspection guidance would be used in conjunction with the nationwide 
regulatory activities plan for fiscal year 2009. Overall, the actions taken 
(and planned to be taken) by TSA are positive developments, although it is 
too soon to determine the extent to which TSA will assess air carriers' 
compliance with the revised watch-list-matching requirements. 

According to TSA officials, there were other benefits stemming from the 
January 2008 special emphasis inspection. For example, officials stated 
that in considering options for corrective actions, TSA consulted with 
representatives from the intelligence community, which is responsible for 
identifying names (and variations of names) 57 for inclusion on the No Fly 
and Selectee lists. According to TSA, these discussions enhanced the 
intelligence community's understanding of how air carriers use the No Fly 
and Selectee lists, and as a result, the intelligence community is better 
positioned to carefully consider which name variations are appropriate for 
being added to the lists and whether these variations would be helpful for 
the purposes of watch-list matching. Further, TSA officials noted that such 
considerations, in turn, could benefit air carriers and the public by limiting 
the number of passengers who are misidentified as being potential 
matches with watch-list records. TSA officials added that insights 
regarding the extent to which name variations exist on the No Fly and 
Selectee lists also have benefited ongoing efforts to design and implement 
the Secure Flight program. Specifically, officials explained that TSA now 
has a fuller understanding of the types of name variations presently 
contained in watch-list records and, in turn, a fuller understanding of what 
types of comparisons Secure Flight should be capable of performing. 



Concluding 
Observations 



Shortcomings that have national security implications exist in the watch- 
list-matching capability of domestic air carriers, as confirmed by the 
results of TSA's recent special emphasis inspection. Specifically, TSA 
found differences among air carriers in the thoroughness and effectiveness 
of their processes for comparing passengers' names with those on the No 
Fly List. A particular concern involves similar-name comparisons. 
However, TSA's April 2008 revision of the No Fly List security directive 



As noted previously, each watch-list record does not necessarily indicate a separate 
individual on the list. Some listed individuals have multiple records attributed to them due 
to the inclusion of known aliases and name variations. 
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establishes a baseline name-matching capability by specifying the types of 
name variations that air carriers' processes must be capable of identifying. 
Effective implementation of the baseline capability should strengthen 
watch-list-matching processes, especially for those air carriers that had 
been using less thorough approaches for identifying similar-name matches. 
Concurrently, revised internal guidance for TSA's inspectors can help 
ensure that compliance decisions are based upon testing and that these 
tests are carried out regularly, using the standards specified within the 
security directives as evaluation criteria. Also, if properly documented in 
inspection reports, the results of these tests could give TSA management 
better information on the quality of watch-list matching being conducted 
by air carriers, thereby improving TSA's monitoring of the overall security 
posture of the aviation sector. At the time of our review, TSA's process for 
revising its guidance was in the initial stages; thus it is too early to 
determine the extent to which updated guidance for principal security 
inspectors and transportation security inspectors would strengthen 
oversight of air carriers' compliance with the security directive 
requirements. Given continued delays in the implementation of the Secure 
Flight program, TSA's oversight of air carriers' compliance with watch-list- 
matching requirements remains an important responsibility. TSA officials 
acknowledge that the baseline capability specified in the revised No Fly 
List security directive and the similar revision planned for the Selectee List 
security directive — while an improvement — does not address all 
vulnerabilities identified by TSA and does not provide the level of risk 
mitigation that is expected to be achieved from Secure Flight. Thus, TSA 
intends to deploy the Secure Flight program beginning in January 2009 so 
that it may implement this more robust matching capability. 



We provided a draft of our restricted report (GAO-08-453SU) to the 
Department of Homeland Security and the Department of Justice for 
review and comment. The Department of Homeland Security had no 
comments. The Department of Justice provided technical comments on 
the restricted version of this report, which we incorporated where 
appropriate. 

We will send copies of this report to the appropriate congressional 
committees; the Secretary of Homeland Security; and the U.S. Attorney 
General. We will make copies available to others upon request. The 
report will also be available at no charge on our Web site at 
http://www.gao.gov. 



Agency Comments 
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If you or your staff have any questions about this report or wish to discuss 
the matter further, please contact me at (202) 512-3404 or 
berrickc@gao.gov. 

Contact points for our Offices of Congressional Relations and Public 
Affairs may be found on the last page of this report. GAO staff who made 
major contributions to this report are listed in appendix III. 




Cathleen A. Berrick 

Director, Homeland Security and Justice Issues 
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Appendix I: Objectives, Scope, and 
Methodology 



To examine the current processes that domestic air carriers use to 
conduct watch-list matching for domestic flights, we addressed the 
following questions: (1) What are TSA's requirements for domestic air 
carriers to conduct watch-list matching for domestic flights? (2) To what 
extent has TSA assessed domestic air carriers' compliance with watch-list- 
matching requirements? 



In addressing the principal questions, we drew upon our previous work 
and reports on aviation security — specifically, reports covering TSA's 
inspection process, Secure Flight, and other passenger prescreening 
programs. We also consulted our most recent reports and testimonies on 
terrorist watch lists. In addition, we reviewed relevant studies conducted 
by other governmental agencies, including the Congressional Research 
Service and the Department of Justice's Office of Inspector General. This 
report is a public version of the restricted report that we provided to 
congressional committees in July 2008. 1 

More details about the scope and methodology of our work to address 
each of the principal questions are presented in the following sections, 
respectively. 



To determine TSA's requirements for air carriers to match passenger 
information against the No Fly List and the Selectee List for domestic 
flights, we assessed two key TSA documents — the No Fly List Procedures 
security directive and the Selectee List Procedures security directive. 2 We 
reviewed versions of these security directives — including the revisions 



1 GAO, Aviation Security: Pending Implementation of Secure Flight, TSA Is Enhancing 
Its Oversight of Air Carrier Efforts to Identify Passengers on the No Fly and Selectee 
Lists, GAO-08-453SU (Washington, D.C.: July 10, 2008). 

2 These directives apply to domestic air carriers — that is, U.S. air carriers that maintain 
security programs in accordance with 49 C.F.R. part 1544. The directives govern watch-list 
matching for flights operating between two points within the United States or its 
territories. Although outside the scope of our review, the directives also apply to domestic 
air carriers' international operations. At the start of our review, we based our analysis on 
the No Fly List Procedures (1544-01-20D) security directive and the Selectee List 
Procedures (1544-01-21E) security directive, both dated July 8, 2004. Over the course of our 
review, TSA first issued revised security directives in 2007 and has undertaken to revise 
them again in April 2008. The 2007 revisions of the No Fly and Selectee list security 
directives (SD 1544-01-20E and SD1544-01-21F, respectively) clarified certain elements of 
the directives but resulted in no substantive changes in the requirements. Generally, in this 
report, we focus on the changes in requirements resulting from revisions undertaken in 
April 2008 (SD 1544-01-20F and anticipated SD 1544-01-21G (Selectee List), respectively). 
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TSAs Requirements for Air 
Carriers to Conduct Watch- 
List Matching for Domestic 
Flights 
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made in April 2008 — to identify applicable requirements for watch-list 
matching. For the purposes of this report, we considered applicable 
requirements to be those that, according to TSA, would be assumed by the 
Secure Flight program, once operational, and those that TSA had itself 
identified for its oversight activities. 3 Thus, we identified the following 
requirements (or key processes) as being within this scope (see table 1, 
which is presented earlier in this report): (1) the retrieval of the No Fly 
and Selectee lists, (2) the matching of passenger and watch-list 
information, (3) the use of the TSA Cleared List, (4) procedures for 
notifying authorities, and (5) keeping appropriate records. 4 

To further our understanding of these requirements, we reviewed TSA 
policies and other guidance applicable to watch-list matching. We also 
interviewed officials from TSA's Office of Security Operations, which had 
primary responsibility for writing the security directives, and officials from 
two TSA offices that collaborated with the Office of Security Operations in 
crafting critical sections of the directives — the Office of Transportation 
Sector Network Management and the Office of Intelligence. To better 
understand TSA's rationale for similar-name-matching requirements as 
well as the challenges associated with name-based matching, we attended 
meetings of the interagency Federal Identity Match Search Engine 
Performance Standards Working Group, which was organized by the 
Terrorist Screening Center to help ensure awareness of best practices with 
regard to identity matching among federal agencies, and spoke with one of 



3 We based our understanding of TSA's planned capabilities for Secure Flight on our 
February 2006 testimony before the Senate Committee on Commerce, Science, and 
Transportation, our most recent, comprehensive testimony on the program when we 
initiated our work in July 2006. See GAO, Aviation Security: Significant Management 
Challenges May Adversely Affect Implementation of the Transportation Security 
Administration's Secure Flight Program, GAO-06-374T (Washington, D.C.: Feb. 9, 2006). 

4 Although addressed in the security directives, other requirements that we excluded from 
our scope involved, for example, procedures involving the screening of employees and 
procedures related to the international operations of domestic air carriers. We did not 
consider requirements for domestic air carriers' international flights as part of our review 
because at the time we were planning our review, TSA intended for Secure Flight to take 
over the watch-list-matching function for only domestic flights. U.S. Customs and Border 
Protection was expected to conduct the watch-list-matching function for flights arriving 
from or departing to locations outside the United States, not Secure Flight. However, in 
February 2008 we reported in testimony that, as agreed to by the respective agencies, TSA 
will also take over the matching of international passengers against the No Fly and Selectee 
lists from U.S. Customs and Border Protection. GAO, Aviation Security: Transportation 
Security Administration Has Strengthened Planning to Guide Investments in Key 
Aviation Security Programs, but More Work Remains, GAO-08-465T (Washington, D.C.: 
Feb. 28, 2008). 
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the group's experts working in the field of name matching. 5 To obtain 
information on the composition and use of the No Fly and Selectee lists, 
we spoke with officials from the Department of Justice's Terrorist 
Screening Center and TSA's Office of Intelligence. Further, to understand 
how TSA compiles and disseminates its Cleared List to air carriers, we 
spoke with officials from the Department of Homeland Security's Traveler 
Redress Inquiry Program (TRIP) and TSA's Office of Transportation 
Security Redress, which share responsibility for managing the TSA 
Cleared List for the current watch-list-matching process. Finally, to 
compare the current watch-list-matching process with that proposed once 
the federal government performs watch-list matching, we reviewed recent 
Secure Flight program documents. 6 

To generally understand how domestic air carriers have responded to 
TSA's requirements, we selected for interviews a nonprobability sample of 
14 air carriers from a TSA-provided list of 95 air carriers that were subject 
to the watch-list-matching security directives for fiscal year 2005. To 
ensure that our sample of air carriers reflected a range of operational 
sizes, we based our selections partly on data from the U.S. Department of 
Transportation, which places air carriers in size categories based on 
operating revenue. Specifically, we selected 8 that were considered 
"major" air carriers, each having more than $1 billion in operating revenue 
in 2005; all but one of these 8 major air carriers flew internationally. In 
addition, we selected 3 air carriers the Department of Transportation 
identified as "national" air carriers, each having more $100 million to 
$1 billion in operating revenue in 2005, and 1 air carrier the department 
identified as a "regional" air carrier, with $100 million or less in operating 
revenue. We also selected two air carriers from the list that were not 
included in the Department of Transportation's revenue groupings, given 
the small scale of their operations, but were identified by the department 
as air carriers that provide commuter service. National, regional, and 
commuter air carriers — which generally provided service covering a 



5 One objective of the Federal Identity Match Search Engine Performance Standards 
Working Group is to provide guidance to improve the effectiveness of the automated 
search engines that federal agencies use for conducting identity matching. The group began 
meeting in December 2005. It included representatives from the departments of Homeland 
Security, State, and Defense; FBI; the intelligence community; and the National Institute of 
Standards and Technology. 

6 Specifically, we reviewed the Secure Flight notice of proposed rulemaking (72 Fed. Reg. 
48,356 (Aug. 23, 2007)) and final concept of operations for Secure Flight (dated Mar. 9, 
2007). We also reviewed our most recent reports and testimonies on the program. 
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geographical area, such as the Pacific Northwest — had comparatively 
smaller business operations. 

In selecting the 14 air carriers, we also considered the number of 
passengers transported. To determine this number, we used the 
Department of Transportation's data for number of revenue passengers 
who enplaned (boarded) domestic air carriers during calendar year 2005 — 
the most recent year for which data were available when making our 
selections in 2006. 7 To the extent possible, we identified the number of 
domestic enplanements for those air carriers required to perform watch- 
list matching in 2005, identified within the previously cited TSA list. 
According to our calculations, the 14 air carriers in our study accounted 
for approximately 70 percent of all passengers who boarded domestic air 
carriers' flights during calendar year 2005, and thus, our selection allowed 
us to understand how watch-list matching was performed for the majority 
of passengers flying domestically in 2005. Although the 14 domestic air 
carriers we selected represent a range in size of air carrier operations and 
transported a majority of passengers that boarded domestic flights in 
calendar year 2005, the results of our interviews are not generalizable to 
all domestic air carriers. 

To help ensure consistency in conducting our interviews with air carriers, 
we developed a data collection instrument with questions focusing on air 
carriers' implementation of certain requirements of the No Fly and 
Selectee list security directives. We conducted four of these interviews in 
person at the air carriers' headquarters and the rest via telephone. In 
addition, to clarify our understanding of air carriers' processes, we 
conducted follow-up phone interviews with four selected air carriers and 
received written answers to our follow-up questions from an additional 
four selected air carriers. The air carrier officials who answered our 
questions generally held positions in corporate security and regulatory 
affairs; however, half of the air carriers also had information technology 
systems specialists participate to answer technical questions related to 
automated name-matching systems. We did not audit or independently 
verify each air carrier's implementation of TSA's security directive 
requirements; rather, our work summarizes the capabilities as reported by 
officials at the 14 air carriers. 



7 Specifically, the data reflect the number of domestic passengers who boarded (enplaned) 
at a flight's point of origin in calendar year 2005. The data include only revenue passengers, 
or passengers from whom the air carrier received payment. As such, the data exclude 
passengers using frequent flier vouchers, infants, air carrier employees, etc. 
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Finally, to understand challenges air carriers have experienced in 
implementing watch-list-matching requirements, we examined TSA's case 
files on all regulatory violations of the No Fly List Procedures and the 
Selectee List Procedures security directives reported since the directives 
were first issued by TSA in 2002 to the time TSA provided us with the data 
in November 2007 — a total of 32 cases. 8 We reviewed these case files, 
which contained documentation and other legal analyses pertaining to 
TSA's inspection findings following the discovery of the violation, to 
determine the nature and causes (i.e., human or electronic) of the 
violations and to identify any patterns among the cases. Finally, to clarify 
the agency's process for investigating and adjudicating security directive 
violations, we spoke with officials from TSA's Office of Chief Counsel. 



Extent to Which TSA Has 
Assessed Domestic Air 
Carriers' Compliance with 
Watch-List-Matching 
Requirements for 
Prescreening Passengers 



To address this objective, we first obtained an overview of TSA's plans and 
guidance for assessing air carriers' compliance with regulatory 
requirements. For instance, to understand the inspection process, the 
focus of inspections, and inspection methods, we reviewed TSA's National 
Inspection Manual, the Principal Security Inspector Handbook, and 
related implementing guidance and policy documents. Further, we 
interviewed or received written responses to our submitted questions from 
the general manager of TSA's Office of Transportation Sector Network 
Management, the two branch chiefs in the office's Commercial Aviation 
Sector, and all nine of the office's principal security inspectors. We 
particularly focused on contacting the principal security inspectors 
because they are responsible for conducting inspections at air carriers' 
corporate security offices (where watch-list-matching policies and 
procedures are formulated) that apply across an air carrier's operations. In 
addition, to obtain information on the creation of inspection plans and 
guidance and the compilation and analysis of inspection data, we spoke 
with individuals in the Office of Security Operations and the Office of 



The earliest case was dated December 3, 2003; the most recent was dated August 24, 2007. 
Because some domestic air carriers that are subject to security directives fly 
internationally, 7 of the 32 cases involved flights arriving from or departing to international 
locations. Although we excluded such flights from our review of watch-list-matching 
requirements, as mentioned previously, we retained these 7 cases in our analysis of 
regulatory violations. We did so because (1) the requirements for air carriers to perform 
watch-list matching for flights involving an international location are, for the most part, the 
same as those for air carrier operations between two points within the United States or its 
territories, and (2) in August 2007, TSA announced that Secure Flight would eventually 
assume watch-list matching for passengers on flights arriving from or departing to 
locations outside the United States. 



Page 41 



GAO-08-992 Aviation Security and Watch List Matching 



Appendix I: Objectives, Scope, and 
Methodology 



Transportation Sector Network Management. Also, to obtain 
management's perspectives on inspections, we spoke with the assistant 
general managers of the Office of Security Operations' Compliance 
Division and its Procedures Division. We also interviewed two federal 
security directors 9 and two transportation security inspectors, also within 
TSA's Office of Security Operations and who were located in the 
Washington, D.C., metropolitan area, on planning and conducting 
inspections. 

After obtaining an understanding of TSA's plans and guidance for 
assessing air carriers' compliance with regulatory requirements, we 
reviewed the results of TSA inspections that are scheduled on a regular 
basis in conjunction with annual inspection plans. In conducting 
inspections each year, TSA's inspectors use an extensive list of inspection 
guidelines (known as PARIS prompts) 10 that cover a broad range of 
applicable topics — including topics outside the scope of our review, such 
as airport perimeter security and cargo security, as well as screening of 
employees and baggage. 11 As presented in table 3, we determined that TSA 
used 11 inspection guidelines during fiscal year 2007 that were relevant to 
the objectives of our review. 12 Of these, guidelines 1, 2, and 6 through 11 
were applicable to inspections conducted by principal security inspectors, 
while guidelines 3 through 5 were applicable to inspections conducted by 
transportation security inspectors. 



Federal security directors are responsible for leading and coordinating TSA security 
activities at airports across the nation. 

10 The Performance and Results Information System (PARIS) is an inspections database 
that assists TSA management by providing factual and analytical information on the 
compliance of TSA-regulated entities. 

11 As mentioned previously, the watch-list-matching requirements relevant to the objectives 
of our review are shown in table 1, which is presented earlier in this report. 

12 TSA provided us with data for 12 inspection guidelines. These 12 are the 11 guidelines 
shown in table 3 — plus the following guideline, which was replaced in March 2007 with 
guideline 4 in table 3: "All passenger names are compared to the most current No Fly and 
Selectee Lists in accordance with the procedures outlined in Security Directive 1544-01-20 
series (No Fly) and Security Directive 1544-01-21 series (Selectee)." Because these two 
guidelines were used for the same purpose but at different times during fiscal year 2007, we 
combined the data associated with each one and treated them as one inspection guideline 
for the purposes of this report. 
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Table 3: Watch-List-Matching Requirements and the Related Inspection Guidelines (Fiscal Year 2007) 


Requirements (key processes) 


Inspection guidelines (prompts) 


Retrieving the No Fly and Selectee lists 


1. 


Procedures are in place to ensure the most recently issued No Fly List is utilized 
within 24 hours of receipt. 




2. 


Procedures are in place to ensure the most recently issued Selectee List is utilized 
within 24 hours of receipt. 


Matching passenger data to No Fly and 
Selectee lists 


3. 


All passenger names are compared to the most current No Fly and Selectee lists in 
accordance with the Private Charter Standard Security Program. 




4. 


The aircraft operator is comparing all passenger names to the most current No Fly 
and Selectee lists in accordance with the procedures outlined in Security Directive 
1544-01-20 series (No Fly) and Security Directive 1544-01-21 series (Selectee). 


Using the TSA Cleared List 


5. 


A passenger identified as a match on the Selectee List is cleared, along with his or 
her accessible property. 


Notifying authorities 


6. 


Procedures are in place to contact the federal security director, local law 
enforcement, FBI, and TSA Office of Intelligence for matches to the No Fly List. 




7. 


Procedures are in place to contact the TSA Office of Intelligence for matches to the 
Selectee List. 


Keeping records 8 


8. 


Records are maintained of all flights operated with passengers who were determined 
by a local law enforcement, U.S. legal attache, or TSA Office of Intelligence not to be 
a match. 



9. Records are maintained of every flight operated with passengers who are designated 
as selectees. 

1 0. Records are maintained of every flight with an individual who is cleared to fly utilizing 
data in the TSA Cleared List including the name of the cleared individual and the 
accepting aircraft operator representative." (No Fly List) 

1 1 . Records are maintained of every flight with an individual who is cleared to fly utilizing 
data in the TSA Cleared List including the name of the cleared individual and the 
accepting aircraft operator representative." (Selectee List) 

Sources: GAO analysis of TSA's security directives and related guidance. 

"Maintaining accurate records, according to TSA officials, provides a starting point for an investigation 
in the event of a terrorist incident. 

"This inspection guideline reflects the current process, which is to use the TSA Cleared List. Security 
directives in effect prior to April 2008 referenced a "cleared column," a format for clearing passengers. 
TSA eventually replaced this format with the Cleared List and updated language in the April 2008 
revision of the No Fly List Procedures security directive to reflect the new process. 

In reference to the 12 inspection guidelines — the 11 guidelines listed in 
table 3 and the 1 guideline discussed in footnote 12 of this appendix — TSA 
queried its PARIS database to identify all inspections of domestic air 
carriers conducted during fiscal year 2007 that used at least one of these 
guidelines. In addition to determining the number of inspections, we 
reviewed the fiscal year 2007 inspection data to calculate compliance 
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rates. 13 We did not evaluate the substantive basis for the inspectors' 
assessment decisions regarding compliance with requirements. 

To determine whether and to what extent TSA's inspectors tested the air 
carriers' capability to conduct exact-name and similar-name matching, we 
also reviewed documentation of testing in a data field (in the PARIS 
database) that allowed inspectors to enter narrative comments regarding 
similar-name matching, among other inspection activities. In doing so, we 
conducted a formal content analysis by having two analysts independently 
review comments in the data field and then resolve any inconsistencies 
between the two sets of analytical observations. Moreover, we submitted 
written questions to each of TSA's nine principal security inspectors 
asking them to describe their practices for testing air carriers' capability to 
identify similar-name variations. 

In contrast to these regular inspections, TSA also conducted a special 
emphasis assessment and a special emphasis inspection, nonroutine 
activities conducted at the direction of TSA headquarters. A special 
emphasis assessment addresses a vulnerability that generally is not tied to 
a regulation, while a special emphasis inspection is tied to a regulatory- 
requirement. TSA provided us information on the scope, methodology, and 
results of a special emphasis assessment that TSA conducted during June, 
July, and September 2005. We reviewed the scope, methodology, and 
results of this assessment with our methodologists and with TSA officials. 
We determined that the sampling and related procedures used for the 
special emphasis assessment were insufficient for providing a reliable 
estimate of the success rate of all attempted matches by air carriers; thus, 
the results cannot be used to infer overall or individual rates of 
compliance with the name-matching requirements in TSA's security 
directives. 

In February 2008, TSA provided us a briefing on the scope and 
methodology of a special emphasis inspection conducted the month 
before in which the similar-name-matching capability of 52 domestic air 
carriers and 31 foreign air carriers was tested. The briefing also covered 
analyses of the results to date of the special emphasis inspection and a 
discussion of the corrective actions that TSA was planning to implement 
to address deficiencies. In April 2008, TSA provided us with an updated 
briefing on its plans for corrective actions. In September 2008, we 



Our calculations were based only on the 12 inspection guidelines relevant to our review. 
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requested information on TSA's progress with these corrective actions. In 
response, TSA provided us the results of a special emphasis assessment 
(conducted during May 20-29, 2008) of seven air carriers' compliance with 
requirements in the April 2008 No Fly List security directive. We did not 
assess the reliability of the data TSA collected during the January 2008 
special emphasis inspection nor the May 2008 special emphasis 
assessment. 



Reliability of Fiscal 
Year 2007 Inspections 
Data 



In assessing the reliability of the fiscal year 2007 data that TSA provided us 
for watch-list-related inspections based on annual inspection cycles, we 
performed electronic testing, discussed the data system and any data 
inconsistencies we found with knowledgeable TSA officials, and reviewed 
existing information about the system. Although we determined that the 
data were reliable for the purposes of this report, we have concerns about 
TSA's process for querying its inspection database, and the potential for 
faulty output. The process is cumbersome and prone to user error due, in 
part, to differences that occur in the verbiage of inspection guidelines and 
types of inspections as they are revised over time. 



We conducted this performance audit from July 2006 to September 2008 in 
accordance with generally accepted government auditing standards. Those 
standards require that we plan and perform the audit to obtain sufficient, 
appropriate evidence to provide a reasonable basis for our findings and 
conclusions based on the audit objectives. We believe that the evidence 
obtained provides a reasonable basis for our findings and conclusions 
based on the audit objectives. 
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TSA's watch-list-matching requirements for domestic flights address five 
key process areas: (1) retrieval of the No Fly and Selectee lists, (2) the 
matching of passenger and list information, (3) the use of TSA's Cleared 
List, (4) notification procedures, and (5) record-keeping activities (see 
table l). 1 

To generally understand how TSA's requirements for watch-list matching 
were being implemented, we reviewed documents in which TSA provided 
general information on air carriers' processes. We also interviewed 14 
domestic air carriers with operations ranging in size from international to 
commuter service about their watch-list-matching processes. All 14 air 
carriers were subject to TSA's requirements for comparing passenger 
information with records on the No Fly and Selectee lists and the TSA 
Cleared List. 2 We asked each of the 14 to describe their processes for 
meeting TSA's requirements. 3 The air carriers' implementation of these 
requirements can be discussed in reference to three time periods — before 
passenger check-in, at passenger check-in, and after passenger check-in — 
as reflected in the following sections, respectively, and as illustrated in 
figure 1. 



1 To identify these requirements, we reviewed the No Fly List Procedures and Selectee List 
Procedures security directives (series SD 1544-01-20 and SD 1544-01-21, respectively). This 
report discusses only the requirements within the two security directives pertaining to 
domestic flights (defined as flights occurring between points within the United States and 
its territories), though these same requirements generally apply to the international flights 
of both domestic and foreign air carriers. For more information on how we identified 
requirements for watch-list matching, see appendix I. 

2 For information on our methodology for selecting the 14 air carriers and conducting the 
interviews, see appendix I. 

3 The implementation methods described in this appendix are based on descriptions 
obtained from the 14 air carriers. We did not undertake audits of the air carriers' processes 
to confirm that the processes functioned as described in the interviews. Specifically, we 
asked air carriers questions on methods for securing the most recent No Fly and Selectee 
lists, executing comparisons within required time frames, determining valid matches, and 
implementing required notification and reporting procedures. 
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Before Passenger 
Check-in: Retrieving 
the No Fly and 
Selectee Lists and 
Executing Name 
Comparisons 



The 14 air carriers told us that they obtain new versions of the No Fly and 
Selectee lists through one or both of the following methods (1) assigning 
an employee to monitor TSA's Web board for new postings at certain 
intervals throughout the day, and (2) receiving an e-mail message from 
TSA to the respective air carrier's security staff informing them of new No 
Fly and Selectee lists. Also, all 14 air carriers reported using passenger 
name record (PNR) data — data collected from the passenger at the time a 
reservation is made — to make comparisons against the No Fly and 
Selectee lists. Specifically, the air carriers said that they have implemented 
procedures to execute comparisons of PNR and watch-list data prior to 
scheduled flight departure. Most of the air carriers told us they do this by 
using computerized matching programs that automatically execute 
comparisons. 



Because the 14 air carriers we interviewed did not collect date of birth (an 
identifying data element that air carriers receive on the No Fly and 
Selectee lists) within PNR data, this information generally was not 
available for matching purposes prior to check-in. However, as discussed 
later in this appendix, several air carriers reported developing systems 
capable of accessing passenger date-of-birth information collected and 
stored outside of PNR data for use in comparisons conducted prior to 
check-in, but this information was not available for all of their passengers. 
Thus, the 14 air carriers we spoke with were limited to performing name- 
only comparisons — that is, comparisons of passenger names with names 
on the No Fly and Selectee lists — prior to check-in for at least some, if not 
all, passengers. All 14 air carriers we spoke with reported conducting 
comparisons to identify exact-name matches of passengers and watch-list 
names. However, not every air carrier reported conducting comparisons to 
identify similar-name matches. 
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At Passenger Check- 
in: Completing 
Comparisons of 
Passenger and Watch- 
List Information and 
Using TSA's Cleared 
List 



In accordance with TSA requirements, air carriers are to collect additional 
identifying information at check-in to assist in identifying passengers who 
are matches with information on the No Fly or Selectee lists. Air carriers 
collect additional identifying information at check-in only for those 
passengers identified as potential matches to the No Fly or Selectee lists 
through the name-only comparisons they conduct prior to check-in. To 
prevent individuals who are potential matches from checking in by other 
means, such as using Internet or airport kiosk check-in, air carriers with 
automated systems place an automatic "lock" on boarding passes (see 
fig. I). 4 By doing so, the air carriers force all potentially matched 
passengers to check in at the ticket counter, where an agent is to collect a 
valid form of identification with date of birth (typically, a government- 
issued identification document such as a driver's license or passport) to 
complete the comparison of passenger and watch-list information. 



To check the potentially matched passenger's date of birth information 
against the No Fly and Selectee lists, most of the 14 air carriers we 
interviewed reported comparing the two dates manually, and the other air 
carriers reported keying the passenger's date of birth into a computer 
system that would automatically execute the comparison. 5 The 14 air 
carriers reported that if they determine that the dates of birth do not 
match, they unlock the boarding pass without consulting TSA, in 
accordance with TSA requirements, thereby allowing the passenger to 
continue the boarding process (see fig. 1, post-check-in number l). 6 
However, if a passenger's date of birth matches with that of an individual 
on the No Fly or Selectee lists, the 14 air carriers said that they consider 



4 The one air carrier in our review without an automated system reported requiring all 
passengers, regardless of whether they were a potential match, to check in at the ticket 
counter. To identify those passengers who should submit additional information for further 
comparison against the No Fly and Selectee lists at check-in, this air carrier reported 
having its employee in charge of watch-list matching make a written notation next to the 
name of all identified potential matches on a printed list of passengers with reservations. 

5 In addition, to check potentially matched passenger information against the No Fly and 
Selectee lists, three air carriers reported that they had developed kiosks with capabilities to 
read electronic date of birth information from certain forms of identification that are 
machine readable. 

6 After this point, the passenger generally experiences no further inconvenience due to 
watch-list matching. However, the passenger may be selected for enhanced checkpoint 
screening as a result of the Computer Assisted Passenger Prescreening System (CAPPS) — 
an electronic application that selects individuals for enhanced screening at the passenger 
checkpoint based on certain travel characteristics identified by TSA as indicating potential 
risk. 
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the passenger to be a match and followed the procedures outlined in TSA's 
security directives for handling matches to the No Fly or Selectee lists (see 
fig. 1, post-check-in numbers 2 and 3). 



Figure 1 : Overview of the Current Passenger Watch-List-Matching Process 
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Also, 10 air carriers reported using the TSA Cleared List to identify and 
clear passengers misidentified as a match to the No Fly List or the Selectee 
List, generally at the time of check in. The other 4 air carriers reported not 
using the list — despite TSA's requirement that all air carriers do so. In 
addition, of the 10 air carriers that reported using the cleared list, 2 
reported using the list in conjunction with their independently developed 
processes to "pre-clear" individuals (discussed below). Development of 
such processes was undertaken to allow air carriers to identify and clear 
misidentified passengers without requiring them to check in at the ticket 
counter. Specifically, 11 of the 14 air carriers we interviewed reported that 
individuals on the TSA Cleared List still must approach the ticket counter 
at check in. 7 Consequently, 6 of the 14 air carriers that we interviewed 
reported developing alternative clearance processes to decrease the 
number of potentially matched individuals who are required to check in at 
the ticket counter. These 6 carriers explained that their internally 
developed clearance processes operate by using additional data sources, 
such as passenger information collected in frequent flier databases, to 
resolve potential matches prior to check in. For example, if an air carrier 
collected date of birth within its frequent flier database, its internal 
clearance system would compare the date of birth of a potentially matched 
passenger who had entered a frequent flier number when making a 
reservation with the date of birth of the respective individual on the No Fly 
List or the Selectee List. 8 



7 These individuals are required to check in at the ticket counter because the air carrier 
must confirm that the passenger is the cleared individual by comparing the passenger's 
legal identifying documentation with the TSA Cleared List. 

8 Air carriers with frequent flier programs generally have the capability to collect a frequent 
flier number within the PNR; therefore, unlike date of birth information, frequent flier 
numbers are available to air carriers prior to a passenger's arrival at check-in and can be 
used to assist in the confirmation of a passenger's identity because of the presence of date 
of birth information in the passenger's frequent flier account. 
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For match determinations made at the time of passenger check in, TSA's 
No Fly and Selectee list security directives require that air carriers follow 
certain notification and record-keeping procedures. With regard to 
notification procedures: 

• If the air carrier identifies a passenger as a potential match to the No 
Fly List, the air carrier must contact both the applicable federal 
security director and the appropriate law enforcement officer. Then, if 
the law enforcement officer confirms that the passenger is a match, the 
air carrier is to contact the local Federal Bureau of Investigation (FBI) 
field office and TSA's Office of Intelligence. 

• If the air carrier identifies a passenger as a potential match to the 
Selectee List, the air carrier must mark the passenger's boarding pass 
to indicate to checkpoint screeners that the passenger should be 
subject to enhanced checkpoint screening. Also, the air carrier must 
notify TSA's Office of Intelligence that the passenger has been matched 
with the Selectee List. 

With regard to record-keeping procedures, TSA's security directives 
require that air carriers maintain a record of (1) all passengers cleared 
using the TSA Cleared List, (2) all flights that had potentially matched 
passengers who were determined by local law enforcement not to be a 
match to the No Fly List, and (3) all passengers identified as matches with 
the Selectee List. 

Generally, the 14 air carriers told us that they followed the notification and 
record-keeping requirements specified in TSA's security directives, but 
reported having different procedures in place to implement these 
requirements. For example, upon identifying a potential match to the No 
Fly List, 5 air carriers reported requiring their ticket agents to notify their 
respective air carrier's ground security coordinator, who would then make 
the necessary calls to the applicable TSA federal security director and to 
local law enforcement. Three other air carriers reported requiring that 
ticket agents contact security staff at a centralized call center, and these 
staff would then make the necessary notifications. 9 In addition, some of 
the carriers reported using some slight deviations from the stated 
requirements. For example, rather than notifying the local FBI field office 



After Passenger 
Check in: 
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Another air carrier reported requiring the ticket agent to make these notifications; the 
other five air carriers we interviewed did not discuss this aspect of the watch-list-matching 
process. 
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and TSA's Office of Intelligence of a match only after a local law 
enforcement officer has confirmed the match, 8 air carriers reported 
contacting TSA's Office of Intelligence for every passenger whose 
information matched the No Fly List, regardless of the local law 
enforcement officer's input. 10 



Two air carriers reported that (per the security directive requirement) they waited for 
local law enforcement officer confirmation before calling the FBI field office or TSA's 
Office of Intelligence. One air carrier reported that it could not answer the question; that is, 
having never identified an individual as a name and date of birth match to the No Fly List, 
the air carrier could not say what its actions would be. During our interviews, three air 
carriers did not discuss this aspect of the watch-list-matching process. 
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